Notice the frame.
The visible stories are a selected group.
Look beyond the frame.
CONCEPTS / MEASUREMENT & FRAMING
Selection and survivorship
Who enters the story shapes what the story can tell us.
Follow the idea.
A concept, in three framesRestore the population.
Bring the excluded projects back into view.
Compare the definitions.
A share of successes is not a success rate.
Who enters a sample affects what that sample can tell us.
Selection bias concerns systematic distortion associated with how observations enter or remain in an analysis. Survivorship is a particular selection: we look at what remained, often without seeing what did not.
A feature of successes is not a success rate
In our constructed example, 20 of 30 successful projects launched early: 66.7%. But 80 projects tried that strategy, and 20 reached the goal: 25%. The numerator is the same. The questions and denominators differ.
20 / 30 among successes
20 / 80 among attempts
20 / 80 among attempts
A complete count still has limits
The example supplies known outcomes for every project. Real excluded cases may have unknown outcomes. Adding known unsuccessful cases does not solve every selection problem. These descriptive rates also leave strategy choice and other causes unexplained.
Source and scope
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004). A structural approach to selection bias. Epidemiology, 15(5), 615–625.
This methodological source distinguishes selection mechanisms in causal analysis. Our project counts are original constructed data; this instrument does not implement the paper’s causal diagrams or adjustment methods.