#19. Across many scientific fields, roughly half of the research findings established over the past century – hundreds of thousands of results – might simply be untrue, and the rest might me much less important than previously thought. At the same time, more and more people trying to find the scientific literature themselves to interpret it. This seems like an interesting combination, which I discuss with philosoper of science Remco Heesen.
Timeline
03:00 What does a (single) scientific study reporting a "significant result" tell you?
06:00 The replication crisis
09:00 The holy grail of science: the p-value (or: how most scientists think about evidence and what you can conclude from it)
13:00 P-hacking
15:00 Why whether some study's result is (probably) Really True, depends on doing only study at a time
19:00 Salami publishing and wrong economic and reputational incentives in the academic publication and career systems
25:00 Solutions: preregistration, publishing negative results, opening up peer review
More info
- Remco Heesen: website, twitter, UWA pagina, RUG pagina
- Maarten van Doorn: website, substack, twitter, RU pagina
- Prinz, F., Schlange, T., & Asadullah, K. (2011). Believe it or not: How much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery, 10, 712. doi:10.1038/nrd3439-c1
- John Ionnadis - "Why Most Published Research Findings Are False"
- Remco Heesen - Why the Reward Structure of Science Makes Reproducibility Problems Inevitable
- Nature article over de replicatiecrisis en priming
- Peerreview (Wikipedia)
- Preregistration (Wikipedia)