Statistical & Psychometrics
PABAK (Prevalence-Adjusted Bias-Adjusted Kappa), PI & BI Formula
Report Ted Byrt’s (1993) clinical diagnostic trio—PABAK = 2 * p_o - 1, Prevalence Index PI = |a - d| / N, and Bias Index BI = |b - c| / N—on 2×2 observer reliability studies.
Why reporting Cohen’s κ alone without Byrt’s PABAK, PI, and BI misleads clinical reviewers!
Ted Byrt, Janet Bishop, and John B. Carlin (Journal of Clinical Epidemiology, 1993) proved that two distinct forces distort Cohen’s kappa on 2×2 diagnostic tables [a, b; c, d]: Prevalence Index PI = |a - d| / N (imbalance between positive and negative agreements, lowering κ) and Bias Index BI = |b - c| / N (asymmetry in rater thresholds, inflating κ)! Replacing skewed marginals with balanced 50/50 reference prevalence yields Prevalence-Adjusted Bias-Adjusted Kappa: PABAK = 2 * p_o - 1, where observed agreement p_o = (a + d) / N!
Reporting all three indices side by side is recommended by EQUATOR/GRRAS reliability guidelines: on [a = 82, b = 6, c = 4, d = 8] (N = 100, p_o = 0.90), PABAK = 0.8000 alongside PI = 0.7400 (high prevalence skew) and BI = 0.0200 (minimal threshold bias)!
| 2×2 Observer Table [a, b, c, d] | Observed Agreement p_o | PABAK (2*p_o - 1) | Prevalence Index PI |
|---|---|---|---|
| [82, 6, 4, 8] (N = 100) | 0.9000 | 0.8000 | 0.7400 (High Skew Disclosed) |
| [82, 15, 15, 8] (N = 120, Higher Disagreement) | 0.7500 | 0.5000 | 0.6167 |
| Negative or Zero Total Count | Undefined | "Invalid 2x2 Counts" | Input Guard |
Prevalence-Adjusted Bias-Adjusted Kappa (PABAK = 2 * p_o - 1)
Evaluates 2 * ((a + d) / N) - 1 on [-1, +1] across 2×2 counts B2:B5.
=IF(OR(MIN(B2:B5) < 0, SUM(B2:B5) <= 0), "Invalid 2x2 Counts", ROUND(2 * ((B2 + B5) / SUM(B2:B5)) - 1, 4))Byrt Prevalence Index PI = |a - d| / N
Quantifies positive-vs-negative agreement skew on [0, 1].
=IF(OR(MIN(B2:B5) < 0, SUM(B2:B5) <= 0), "Invalid 2x2 Counts", ROUND(ABS(B2 - B5) / SUM(B2:B5), 4))Try the example
Use the original values or change them to test your understanding.