Causal Inference & Survey Statistics
Horvitz-Thompson vs Hajek Inverse Probability Weighting (IPW) ATE
Compare unnormalized Horvitz-Thompson IPW against self-normalized Hajek IPW in Excel and Arte Sheets to eliminate finite-sample baseline-level distortion and audit propensity positivity.
See in 6 rows why unnormalized Horvitz-Thompson IPW fails location invariance
In observational studies with propensity score e_i = Pr(D_i = 1 | X_i) in (0, 1), classical Horvitz-Thompson (1952) IPW divides weighted totals by nominal sample size N: ATE_HT = (1/N) * SUM(D_i * Y_i / e_i) - (1/N) * SUM((1 - D_i) * Y_i / (1 - e_i)). Whenever finite-sample inverse-propensity weights SUM(D_i / e_i) deviate from N, adding a constant baseline level (such as +100) to every outcome Y_i wildly distorts Horvitz-Thompson! Hajek (1971) normalizes each arm by its own realized weight sum SUM(D_i / e_i) and SUM((1 - D_i) / (1 - e_i)), guaranteeing exact location invariance.
Test this dramatic contrast on N = 6 subjects where every control (D = 0) has Y = 100 and every treated subject (D = 1) has Y = 110 — a deterministic +10.0000 effect! Because one treated subject has low propensity e_3 = 0.20 (weight 5.0), treated weights sum to 2 + 2 + 5 = 9.0 instead of N = 6: Horvitz-Thompson hallucinates ATE_HT = (990 / 6) - (600 / 6) = +65.0000, whereas Hajek divides 990 by 9.0 and 600 by 6.0 to recover exact ATE_HJ = +10.0000!
| IPW Estimator | Denominator Rule | Result on Deterministic +10 Data |
|---|---|---|
| Horvitz-Thompson (Unnormalized) | Divides both arms by nominal N = 6 | +65.0000 (Severe Level Distortion!) |
| Hajek Self-Normalized IPW | Divides by SUM(D/e)=9 and SUM((1-D)/(1-e))=6 | +10.0000 (Exact True Effect) |
| Positivity Overlap Guard | Requires 0 < MIN(e) and MAX(e) < 1 | Flags Extreme Propensity Tail |
Hajek Self-Normalized IPW Average Treatment Effect Formula
Compute row-level treated weight w_1 = D_i / e_i (E2:E7) and control weight w_0 = (1 - D_i) / (1 - e_i) (F2:F7), then form SUMPRODUCT(w_1, Y) / SUM(w_1) - SUMPRODUCT(w_0, Y) / SUM(w_0) with positivity checks.
=IF(OR(MIN(C2:C7) <= 0, MAX(C2:C7) >= 1, SUM(E2:E7) <= 0, SUM(F2:F7) <= 0), "Positivity Violation: 0 < e < 1", ROUND(SUMPRODUCT(E2:E7, D2:D7) / SUM(E2:E7) - SUMPRODUCT(F2:F7, D2:D7) / SUM(F2:F7), 4))Unnormalized Horvitz-Thompson IPW Comparison Formula
Keep Horvitz-Thompson side-by-side during peer review so analysts verify whether SUM(E2:E7) / ROWS(D2:D7) deviates substantially from 1.00.
=IF(OR(MIN(C2:C7) <= 0, MAX(C2:C7) >= 1), "Positivity Violation: 0 < e < 1", ROUND((SUMPRODUCT(E2:E7, D2:D7) - SUMPRODUCT(F2:F7, D2:D7)) / ROWS(D2:D7), 4))Try the example
Use the original values or change them to test your understanding.