Psychology One-page note
The Replication Crisis in One Page
Some of psychology's most famous findings didn't hold up when scientists ran them again. Here's what happened and how to spot a solid claim. Save this page.
Good to know
Open Science Collaboration 2015; Many Labs 2 (Klein et al. 2018); Camerer et al. 2018; Hagger et al. 2016; Simmons et al. 2011.
What's on the page
The note as text, so you can read it, copy it or use a screen reader.
What it is
- Rerun a published study: same result?
- Many famous findings didn't repeat
- Psychology tested itself in the 2010s
The big tests
- 2015: 100 studies rerun
- 97% "worked" first time, 36% on rerun
- 2018: 28 findings, 36 countries: about half
- 2018: 21 Nature & Science studies: 13 repeated
Effects shrank too
- Rerun effects about half the original size
- Ego depletion: 23 labs, one test, found almost nothing
Why it happened
- Flexible analysis can make chance look real
- Some original effects were inflated
- Some originals were false positives
How to judge a claim
- Has it been repeated by others?
- Big sample? Pre-registered?
- One eye-catching study isn't proof
- Failing once doesn't prove it's false
Sources
- Open Science Collaboration (2015): Estimating the reproducibility of psychological science. Science
- Klein, R. A. et al. (2018): Many Labs 2: Investigating variation in replicability across samples and settings. AMPPS
- Camerer, C. F. et al. (2018): Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour
- Hagger, M. S. et al. (2016): A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science
- Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011): False-positive psychology. Psychological Science
- Forsell, E. et al.: Predicting replication outcomes in the Many Labs 2 study (PsyArXiv)
How we checked this
Every fact on this note comes from the 6 sources listed here, checked against official or peer-reviewed sources where possible. AI tools help with drafting; a person checks every fact against the sources. How we research
Spotted a mistake? Tell us and we'll fix it and log it on the corrections page.