Three statistical traps you can run with your own eyes: survivorship bias (the WW2 planes), Simpson's paradox (the kidney-stone study where one treatment wins overall yet loses everywhere), and regres.
Exhibit A ยท Survivorship bias โ the planes that never came back
Armour the planes that returned
WW2, statistician Abraham Wald. Engineers counted bullet holes on the bombers that made it home and proposed armouring exactly where the holes were. Wald said: armour the places with the fewest holes โ because planes hit there never came back to be counted. Run the sorties and watch the gallery lie to you.
0 planes have come home. Every plane in the gallery is a survivor.
The trap. Engines take the fewest recorded hits โ because an engine hit is fatal and the planes that took one are missing from your data. The visible histogram is selected by survival. This exact fallacy is why failed products, lucky traders and "overnight success" stories feel so much more common than they are.
Exhibit B ยท Simpson's paradox โ the treatment that wins by losing
The kidney-stone study (real data, 1986)
Two treatments for kidney stones. Treatment B has a better overall success rate โ and is worse in every single type of case. Both are true at once. Toggle the view and watch the winner flip. This is not a trick of rounding; it is arithmetic.
Why it happens. Treatment B was given mostly to the easy cases (small stones), so its good overall average just reflects an easier patient mix. When you compare like-for-like โ small with small, large with large โ A wins both. Whenever you see "our patients did better than theirs," ask: were the patients actually the same?
Exhibit C ยท Regression to the mean โ the curse of the hot streak
Champions get worse. Mathematically.
Every sport has a "Sports Illustrated curse": great first halves are followed by disappointing ones. It isn't a curse, and it isn't pressure. It's arithmetic โ any extreme result is part skill, part luck, and the luck doesn't repeat. Run the season:
The lesson. Judge the process, not the streak. The player whose first half was a fluke "crashes" back toward their true ability โ the player whose second half regressed may have been the better one all along. Bookies, coaches and hospitals that forget this make real money and real mistakes.
Kidney-stone data: Charig, Webb, Payne & Wickham, British Medical Journal 1986. The plane
story is the standard account of Abraham Wald's work at the Statistical Research Group 1943. Simulation
mechanics on this page are intentionally simple so the arithmetic is visible โ no black box.