Get the latest tech news
The dead weight loss of strictly isotonic regression
Calibration aligns model scores with event frequencies. For a binary outcome $Y\in{0,1}$ and a score $S$, the calibration function is $g(s)=\mathbb{E}[Y\mid S=s]$. Post hoc calibration estimates $g$ on a holdout set and applies the estimate to future scores. A common difficulty
Isotonic regression, implemented by the Pool Adjacent Violators Algorithm, treats each observed inversion as miscalibration and merges neighboring regions until the fitted function is nondecreasing. Fourth, fit a smooth monotone model, such as a shape-constrained spline, and test whether the average slope over each step's range is distinguishable from zero using cross-fitted bootstrap intervals. When differentiability is important, monotone splines, for example, I-splines or shape-constrained generalized additive models, provide smooth increasing calibrators with good interpretability.
Or read this on Hacker News