Statistical & Clinical Analysis
Conger’s Multi-Rater Generalized Kappa (κ_C): Rater-Specific Marginal Agreement
Calculate Anthony Conger’s (1980) exact multi-rater kappa κ_C = (P_o - P_e,C) / (1 - P_e,C) across fixed raters with distinct positive-call rates—and contrast against Fleiss pooled P_e.
Why Fleiss’s kappa assumes random interchangeable raters—whereas Conger’s κ_C generalizes Cohen’s κ to a fixed panel!
When the SAME fixed panel of m = 3 radiologists rates every scan, Fleiss’s kappa pools all raters’ positive rates into p̄ = (p_1 + p_2 + p_3)/3 and computes P_e,Fleiss = p̄^2 + (1 - p̄)^2—ignoring systematic strict-vs-lenient rater bias! Anthony J. Conger (1980, Psychological Bulletin) generalized Cohen’s pairwise cross-product P_e to m fixed raters by averaging exact pairwise chance agreement [p_a*p_b + (1 - p_a)*(1 - p_b)] across all m*(m - 1)/2 rater pairs!
With observed pairwise agreement P_o = 0.78 and individual positive rates p_1 = 0.25 (strict), p_2 = 0.55 (lenient), and p_3 = 0.40 (balanced), true pairwise chance agreement is P_e,Conger = 0.5050, yielding κ_Conger = 0.5556 (versus Fleiss κ = 0.5417 where pooled p̄ = 0.40 overstates chance agreement at 0.5200)!
| Panel Agreement Scenario (p_1 = 0.25, p_2 = 0.55, p_3 = 0.40) | Conger Pairwise P_e,C | Conger Multi-Rater κ_C | Fleiss Pooled κ Comparison |
|---|---|---|---|
| Baseline Observed P_o = 0.78 | 0.5050 | 0.5556 (Moderate) | Fleiss κ = 0.5417 (P_e = 0.5200) |
| High Concordance P_o = 0.90 | 0.5050 | 0.7980 (Substantial–Almost Perfect) | Reflects fixed-panel bias penalty |
| Out-of-Bounds Probability (< 0 or > 1) | Error Guard | "Probabilities Must Be in [0, 1]" | Probability Audit Guard |
Conger Exact Multi-Rater Kappa κ_C = (P_o - P_e,C) / (1 - P_e,C)
Evaluates Conger’s generalized kappa from observed pairwise agreement B2 and exact 3-rater pairwise chance helper B6.
=IF(OR(B2 < 0, B2 > 1, MIN(B3:B5) < 0, MAX(B3:B5) > 1, B6 >= 1), "Probabilities Must Be in [0, 1]", ROUND((B2 - B6) / (1 - B6), 4))Conger Pairwise Chance Agreement P_e,C vs. Fleiss Pooled Kappa
Outputs exact pairwise chance expectation P_e,C across rater pairs (1,2), (1,3), and (2,3) in B6.
=IF(OR(B2 < 0, B2 > 1, MIN(B3:B5) < 0, MAX(B3:B5) > 1), "Probabilities Must Be in [0, 1]", ROUND(B6, 4))Try the example
Use the original values or change them to test your understanding.