display | more...

Survivorship bias is a subtle type of sampling bias, in which the population sampled is distorted because non-existent members are ignored. Normally this is fine, but sometimes non-existent members are non-existent in highly relevant ways.

The most popular example of this is Abraham Wald's story of the US military trying to design safer warplanes. The military started the process by looking at returning airplanes, and counting up which areas of the planes had the most bullet holes, planning to add additional armor to those places. Wald noted that this was backwards, and what they were measuring were the places that a plane could be shot without falling out of the sky; instead, they needed to look at the spots where no bullet holes had been found -- the planes hit in those spots never made it back to base.

The survivorship bias is often relevant in everyday life, and is perhaps more often a cognitive bias than a statistical one. When you hear about the diet that worked miracles, the person who drove drunk and it was fine, and the person who was diagnosed with terminal cancer but was saved through the power of yoga -- all of these are reported because reports of people dying of cancer despite taking yoga classes are not newsworthy, people who drove drunk and crashed the car tend not to report on the matter, and people who gain weight on a diet usually do not brag about that diet.

However, survivorship bias is currently a pressing matter in the social sciences, as it has become apparent that it operates heavily in academic journals. Studies that give interesting results are published, while those that do not are not -- giving the false impression that some results have been universally and reliably confirmed.


Log in or register to write something here or to contact authors.