Biostatistics & Multiple Testing
Benjamini-Hochberg FDR vs Holm-Bonferroni Threshold Formula
Compare Benjamini-Hochberg step-up False Discovery Rate critical values (i/m)*q against Holm-Bonferroni step-down Family-Wise Error Rate cutoffs alpha/(m - i + 1) across sorted p-values.
Understand step-DOWN stopping (Holm) vs step-UP largest-rank discovery (Benjamini-Hochberg)!
Sorting m hypothesis p-values ascending p_(1) <= p_(2) <= ... <= p_(m) unlocks far higher statistical power than rigid Bonferroni alpha / m — yet Holm (1979) and Benjamini-Hochberg (1995) work in OPPOSITE directions! Holm controls Family-Wise Error Rate (FWER) step-DOWN: compare p_(i) to alpha / (m - i + 1) starting at i = 1 and STOP at the very first failure. Benjamini-Hochberg controls False Discovery Rate (FDR) step-UP: compare p_(i) to (i / m) * q, find the LARGEST rank k_max that passes, and reject ALL ranks 1..k_max (even if an earlier rank slightly exceeded its own line)!
Verify on m = 5 sorted p-values B2:B6 = {0.004, 0.013, 0.022, 0.038, 0.065} at target q = alpha = 0.05 (G2): Holm cutoffs C2:C6 = {0.0100, 0.0125, 0.0167, 0.0250, 0.0500} stop at Rank 2 (0.013 > 0.0125), yielding 1 discovery. Benjamini-Hochberg cutoffs D2:D6 = {0.0100, 0.0200, 0.0300, 0.0400, 0.0500} pass through Rank 4 (0.038 <= 0.0400), yielding 4 discoveries! Even if Rank 2 shifts to 0.021 (> 0.0200), BH step-up STILL rejects Ranks 1..4 because Rank 4 passes!
| Sorted Rank i (m = 5, q = 0.05) | Observed p_(i) | Holm Cutoff vs BH Cutoff |
|---|---|---|
| Rank 1 | 0.0040 | 0.0100 (Pass) vs 0.0100 (Pass) |
| Rank 2 | 0.0130 | 0.0125 (STOP!) vs 0.0200 (Pass) |
| Rank 3 | 0.0220 | 0.0167 (Blocked) vs 0.0300 (Pass) |
| Rank 4 | 0.0380 | 0.0250 (Blocked) vs 0.0400 (k_max = 4!) |
| Rank 5 | 0.0650 | 0.0500 (Blocked) vs 0.0500 (Fail) |
Benjamini-Hochberg Step-Up Total Discoveries k_max Formula
Verify ascending p-value order first, then find the largest rank i in 1..5 satisfying p_(i) <= (i / 5) * q.
=LET(q_fdr, G2, IF(OR(q_fdr <= 0, q_fdr >= 1, B2 < 0, B6 > 1, B2 > B3, B3 > B4, B4 > B5, B5 > B6), "Require Sorted p in [0,1]", MAX(IF(B2 <= 0.2 * q_fdr, 1, 0), IF(B3 <= 0.4 * q_fdr, 2, 0), IF(B4 <= 0.6 * q_fdr, 3, 0), IF(B5 <= 0.8 * q_fdr, 4, 0), IF(B6 <= q_fdr, 5, 0))))Holm-Bonferroni Step-Down Sequential Discoveries Formula
Enforce sequential step-down stopping: Rank i can be rejected under Holm if and only if EVERY earlier rank 1..i also passed its Holm cutoff alpha / (m - j + 1).
=LET(alp_f, G2, IF(OR(alp_f <= 0, alp_f >= 1, B2 > B3, B3 > B4, B4 > B5, B5 > B6), "Require Sorted p in [0,1]", IF(B2 > alp_f / 5, 0, IF(B3 > alp_f / 4, 1, IF(B4 > alp_f / 3, 2, IF(B5 > alp_f / 2, 3, IF(B6 > alp_f, 4, 5)))))))Try the example
Use the original values or change them to test your understanding.