Get the latest tech news
Lies. Damned Lies. P-value thresholds
In the era of Big Data, we’ve come to believe that, with enough information, human behavior is predictable. But number crunching can lead us perilously wrong.
David Spiegelhalter, the author of an important and comprehensive new book, “ The Art of Statistics ” (Basic), was one of the statisticians tasked by the ensuing public inquiry to establish whether the mortality rate of Shipman’s patients should have aroused suspicion earlier. The study has a thousand citations, and an accompanying TED talk has amassed more than fifty million views, but the findings have failed to be replicated and are now regarded as a notable example of the flaws in Fisher’s methods. Back when Spiegelhalter was asked to determine whether Dr. Harold Shipman’s mortality rate should have aroused suspicion earlier, he swiftly decided that the standard test of statistical significance would be a “grossly inappropriate” way to monitor doctors.
Or read this on Hacker News