Pinball Quantile Loss & Winkler Interval Score Formula

Stop grading P90 safety-stock quantiles or 90% prediction intervals with symmetric RMSE!

If a supply-chain planner forecasts the 90th-percentile demand quantile q_0.90 to prevent stockouts, grading that forecast with symmetric MAE or RMSE rewards sandbagging toward the median (tau = 0.50)! Roger Koenker and Gilbert Bassett’s (1978) Pinball (Check) Loss L_tau(y, q) = IF(y >= q, tau * (y - q), (1 - tau) * (q - y)) = MAX(tau * (y - q), (tau - 1) * (y - q)) is strictly proper for quantile tau in (0, 1): at tau = 0.90, an under-forecast stockout (y > q) is penalized 9x harder per unit than an over-forecast overstock (y < q)! Similarly, Robert Winkler’s (1972) Interval Score grades a central (1 - alpha) prediction interval [L, U] (e.g., alpha = 0.10 for a 90% interval from P05 to P95) by combining sharpness width (U - L) plus out-of-bounds miss penalties: WS_alpha = (U - L) + (2 / alpha) * MAX(0, L - y) + (2 / alpha) * MAX(0, y - U)!

Verify on Realized_Demand y = 135 (B2), P90_Quantile_Forecast q_tau = 120 (B3, tau = 0.90 in B4), and 90% Prediction Interval [L = 80, U = 125] (B5, B6 with alpha = 0.10 in B7): Pinball Under-Forecast Loss is 0.90 * (135 - 120) = 13.50 (whereas an equal 15-unit over-forecast y = 105 costs only 0.10 * 15 = 1.50!), and Winkler 90% Interval Score combines Width (125 - 80 = 45.00) plus Upper-Tail Breach Penalty (2 / 0.10) * (135 - 125) = 20 * 10 = 200.00 -> Total Winkler Score = 245.00 (lower is better; when y = 110 falls inside [80, 125], penalty is 0 and WS = 45.00!)!

Strictly Proper Asymmetric Pinball Quantile Loss Formula

Validate quantile target 0 < tau < 1. Notice at tau = 0.50, Pinball Loss equals 0.5 * ABS(y - q) (half of MAE).

=LET(y_ac, B2, q_fc, B3, t_qt, B4, IF(OR(t_qt <= 0, t_qt >= 1), "Require 0 < tau < 1", ROUND(MAX(t_qt * (y_ac - q_fc), (t_qt - 1) * (y_ac - q_fc)), 4)))

Winkler (1972) Central (1 - alpha) Prediction-Interval Score Formula

Validate interval ordering L <= U and tail miscoverage rate 0 < alpha < 1 (used in M4/M5 forecasting competitions).

=LET(y_ac, B2, l_bd, B5, u_bd, B6, al_ms, B7, IF(OR(l_bd > u_bd, al_ms <= 0, al_ms >= 1), "Invalid Interval Input", ROUND((u_bd - l_bd) + (2 / al_ms) * MAX(0, l_bd - y_ac) + (2 / al_ms) * MAX(0, y_ac - u_bd), 4)))

Further reading

• Arte Sheets Lesson — Brier Score & Binary Log-Loss Proper Scoring Rules
• Arte Sheets Lesson — MAE, WAPE, and sMAPE Demand Forecast Error Formulas