Statistical & Math
Zipf’s Law Rank-Frequency Power Law & Heaps’ Vocabulary Growth Formula
Calculate Zipf’s rank-frequency count f(r) = f_1 / (r ^ s) for search queries, cities, and natural-language tokens alongside Heaps’ Law distinct vocabulary size V(N) = K * (N ^ beta) for LLM and inverted-index sizing.
Size search-engine inverted indexes, CDN caches, and LLM token vocabularies with power laws!
George Kingsley Zipf observed that in virtually every human language corpus, web-query log, and metropolitan city hierarchy, the frequency f(r) of the r-th ranked item decays inversely with its rank raised to power s (~1.0): f(r) = f_1 / (r ^ s)! When top-ranked token #1 ("the") appears f_1 = 70,000 times with exponent s = 1.02, rank #10 appears 6,684.9 times and rank #100 drops to just 638.4 times—explaining why caching the top 1–5% of hot keys absorbs a massive share of CDN traffic!
Its dual in information retrieval is Harold Heaps’ Law (Herdan-Heaps Law), which predicts how many unique vocabulary types V(N) appear as total corpus size N grows: V(N) = K * (N ^ beta) (typically K = 10–100 and beta = 0.45–0.60 in English text)! At K = 25 and beta = 0.52, a 1,000,000-token corpus yields V = 32,956 unique word types—sub-linear growth that keeps inverted-index dictionaries compact in RAM while never saturating completely!
| Corpus Power-Law Model | Parameters | Evaluated Target | Predicted Count |
|---|---|---|---|
| Zipf’s Law at Rank r = 10 | f1 = 70,000, s = 1.02 | 70000 / (10 ^ 1.02) | 6,684.9 occurrences |
| Zipf’s Law at Rank r = 100 | f1 = 70,000, s = 1.02 | 70000 / (100 ^ 1.02) | 638.4 occurrences |
| Heaps’ Law at N = 1,000,000 Tokens | K = 25, beta = 0.52 | 25 * (1000000 ^ 0.52) | 32,956 unique vocabulary terms |
Zipf’s Law Expected Frequency at Rank r: f(r) = f_1 / (r ^ s)
Validates rank r >= 1 (B3) and positive top frequency f_1 (B2) and Zipf exponent s (B4).
=IF(OR(B2 <= 0, B3 < 1, B4 <= 0), "Invalid Corpus Parameters", ROUND(B2 / (B3 ^ B4), 1))Heaps’ Law Unique Vocabulary Size V(N) = K * (N ^ beta)
Estimates distinct dictionary terms across N total corpus tokens (B5) for scaling constant K (B6) and exponent beta (B7).
=IF(OR(B5 < 1, B6 <= 0, B7 <= 0, B7 >= 1), "Invalid Corpus Parameters", ROUND(B6 * (B5 ^ B7), 0))Try the example
Use the original values or change them to test your understanding.