Statistical
Matthews Correlation Coefficient (MCC) & Cohen's Kappa Confusion Matrix formulas
Compute binary MCC and unweighted Cohen kappa from valid integer counts, distinguish undefined cases and compare asymmetric confusion matrices.
Why 90% Accuracy can be a worthless classifier—and why MCC & Kappa catch it!
Accuracy can hide minority-class errors. MCC uses all four binary confusion-matrix counts; unweighted Cohen kappa compares agreement with agreement implied by the observed marginals. Neither statistic alone establishes usefulness, calibration or fairness.
For the all-negative predictor with both true classes present, the MCC denominator is zero. This exercise returns zero by an explicit software convention, not because zero follows from dividing0by0. Kappa is zero in that example. If all observations belong to one class and predictions match, kappa is undefined because its chance-agreement denominator is zero.
Counts are nonnegative integers and the total must be positive. The formulas are for a binary matrix in the stated TP,TN,FP,FN order. Weighted or multiclass kappa and weighted observations require a different implementation.
| Confusion Matrix Metric | Formula | Strong Model (80, 880, 20, 20) | Dummy All-Negative (0, 900, 0, 100) |
|---|---|---|---|
| Raw Accuracy (TP+TN)/N | (B2+B3) / SUM(B2:B5) | 96.00% (0.9600) | 90.00% (0.9000 — Deceptive!) |
| Matthews Correlation (MCC) | (TP*TN-FP*FN) / SQRT(Marginals) | 70,000 / 90,000 = 0.7778 | 0 by explicit zero-marginal convention |
| Cohen's Chance-Adj Kappa | (p_o - p_e) / (1 - p_e) | (0.96 - 0.82) / 0.18 = 0.7778 | 0 for this matrix |
Guarded Matthews Correlation Coefficient (MCC) on [-1, +1]
Guards degenerate zero-marginal denominators (such as TP + FP = 0 when a model never predicts positive) to return 0 cleanly.
=IF(COUNT(B2:B5)<>4,"Invalid inputs",IF(OR(MIN(B2:B5)<0,SUM(B2:B5)<=0,B2<>INT(B2),B3<>INT(B3),B4<>INT(B4),B5<>INT(B5)),"Invalid inputs",IF((B2 + B4) * (B2 + B5) * (B3 + B4) * (B3 + B5) = 0, 0, ROUND((B2 * B3 - B4 * B5) / SQRT((B2 + B4) * (B2 + B5) * (B3 + B4) * (B3 + B5)), 4))))Cohen's Chance-Corrected Kappa Statistic
Use the algebraically equivalent count-based expression. A zero denominator returns Undefined kappa rather than claiming perfect or zero agreement.
=IF(COUNT(B2:B5)<>4,"Invalid inputs",IF(OR(MIN(B2:B5)<0,SUM(B2:B5)<=0,B2<>INT(B2),B3<>INT(B3),B4<>INT(B4),B5<>INT(B5)),"Invalid inputs",IF((SUM(B2:B5)^2-((B2+B4)*(B2+B5)+(B5+B3)*(B4+B3)))=0,"Undefined kappa",ROUND((SUM(B2:B5)*(B2+B3)-((B2+B4)*(B2+B5)+(B5+B3)*(B4+B3)))/(SUM(B2:B5)^2-((B2+B4)*(B2+B5)+(B5+B3)*(B4+B3))),4))))Try the example
Use the original values or change them to test your understanding.