API Gateways, Networking & Rate Limiting
Token Bucket Max Burst Duration & Volume Formula
Calculate exact unthrottled burst duration T_burst = B / (P - r), total passed burst requests M_max = (B * P) / (P - r), and full recovery time B / r for API token-bucket limiters.
Avoid the #1 API gateway mistake: a B-token bucket passes MORE than B requests during a peak burst!
API gateways (Envoy, NGINX, AWS API Gateway, Stripe, and external OAuth limiters) shape traffic with a Token Bucket of capacity B > 0 tokens refilling continuously at r > 0 tokens/sec. When a client bursts at peak arrival rate P > r requests/sec starting from a full bucket, how many requests pass before the first HTTP 429 throttle? Engineers routinely guess "B requests" — forgetting that tokens KEEP REFILLING at rate r while the bucket drains at net rate (P - r)! Solving B + r * T_burst = P * T_burst gives exact Unthrottled Burst Duration T_burst = B / (P - r) seconds and Total Passed Burst Requests M_max = P * T_burst = (B * P) / (P - r) > B!
Verify by hand on Bucket_Capacity_B = 120 tokens (B2), Refill_Rate_r = 2 tokens/sec (B3 = 120/min sustained), and Peak_Arrival_P = 10 requests/sec (B4): net drain rate is 10 - 2 = 8 tokens/sec, so Max Burst Duration is T_burst = 120 / 8 = 15.0000 seconds, Total Passed Requests before throttling equal 120 + 2 * 15 = 150.00 requests (+30 extra requests refilled during the burst!), and Full Idle Refill Time is 120 / 2 = 60.00 seconds!
| Token-Bucket Metric (B=120, r=2/sec, P=10/sec) | Formula | Verified Output |
|---|---|---|
| Net Drain Rate During Burst | P - r | 8.0000 tokens / sec |
| Unthrottled Burst Duration (T_burst) | B / (P - r) | 15.0000 seconds |
| Total Requests Passed in Burst (M_max) | (B * P) / (P - r) | 150.00 requests (vs Naive 120!) |
| Idle Full-Bucket Refill Time | B / r | 60.00 seconds |
Exact Unthrottled Burst Duration T_burst = B / (P - r) Formula
Branch cleanly when Peak_Arrival_P <= Refill_Rate_r: the bucket never empties and burst duration is infinite ("Infinite: Arrival <= Refill Rate")!
=LET(b_cp, B2, r_rf, B3, p_pk, B4, IF(OR(b_cp <= 0, r_rf <= 0, p_pk <= 0), "Invalid Token Bucket Input", IF(p_pk <= r_rf, "Infinite: Arrival <= Refill Rate", ROUND(b_cp / (p_pk - r_rf), 4))))Maximum Contiguous Burst Requests Passed M_max = (B * P) / (P - r) Formula
Notice that as P -> infinity (instantaneous microsecond batch), M_max -> B; as P approaches r from above, M_max grows arbitrarily large!
=LET(b_cp, B2, r_rf, B3, p_pk, B4, IF(OR(b_cp <= 0, r_rf <= 0, p_pk <= 0), "Invalid Token Bucket Input", IF(p_pk <= r_rf, "Infinite: Arrival <= Refill Rate", ROUND((b_cp * p_pk) / (p_pk - r_rf), 4))))Try the example
Use the original values or change them to test your understanding.