Low replicability can support robust and efficient science.

Stephan Lewandowsky, Klaus Oberauer
Author Information
  1. Stephan Lewandowsky: School of Psychological Science, University of Bristol, 12A, Priory Road, Bristol, BS8 1TU, UK. stephan.lewandowsky@bristol.ac.uk. ORCID
  2. Klaus Oberauer: Department of Psychology, University of Zurich, Zurich, Switzerland. ORCID

Abstract

There is a broad agreement that psychology is facing a replication crisis. Even some seemingly well-established findings have failed to replicate. Numerous causes of the crisis have been identified, such as underpowered studies, publication bias, imprecise theories, and inadequate statistical procedures. The replication crisis is real, but it is less clear how it should be resolved. Here we examine potential solutions by modeling a scientific community under various different replication regimes. In one regime, all findings are replicated before publication to guard against subsequent replication failures. In an alternative regime, individual studies are published and are replicated after publication, but only if they attract the community's interest. We find that the publication of potentially non-replicable studies minimizes cost and maximizes efficiency of knowledge gain for the scientific community under a variety of assumptions. Provided it is properly managed, our findings suggest that low replicability can support robust and efficient science.

References

  1. Science. 2015 Aug 28;349(6251):aac4716 [PMID: 26315443]
  2. Nat Rev Neurosci. 2013 May;14(5):365-76 [PMID: 23571845]
  3. PLoS Med. 2005 Aug;2(8):e124 [PMID: 16060722]
  4. PLoS One. 2014 Jan 17;9(1):e84896 [PMID: 24465448]
  5. Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606 [PMID: 29531091]
  6. Lancet. 2014 Jan 11;383(9912):166-75 [PMID: 24411645]
  7. Psychon Bull Rev. 2014 Apr;21(2):557-61 [PMID: 23975542]
  8. Proc Natl Acad Sci U S A. 2015 Dec 15;112(50):15343-7 [PMID: 26553988]
  9. Psychon Bull Rev. 2009 Apr;16(2):225-37 [PMID: 19293088]
  10. Perspect Psychol Sci. 2012 Nov;7(6):555-61 [PMID: 26168112]
  11. Perspect Psychol Sci. 2016 Sep;11(5):664-691 [PMID: 27694463]
  12. Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2607-2612 [PMID: 29531092]
  13. Psychon Bull Rev. 2016 Oct;23(5):1631-1638 [PMID: 27068542]
  14. PLoS Biol. 2013 Oct;11(10):e1001675 [PMID: 24115908]
  15. Perspect Psychol Sci. 2014 Jan;9(1):59-71 [PMID: 26173241]
  16. Psychon Bull Rev. 2012 Apr;19(2):151-6 [PMID: 22351589]
  17. PLoS One. 2019 Jan 2;14(1):e0208631 [PMID: 30601826]
  18. Front Psychol. 2015 Apr 24;6:494 [PMID: 25964771]
  19. Psychon Bull Rev. 2019 Oct;26(5):1596-1618 [PMID: 31515732]
  20. Nat Hum Behav. 2018 Sep;2(9):637-644 [PMID: 31346273]
  21. Perspect Psychol Sci. 2012 Nov;7(6):585-94 [PMID: 26168115]
  22. Lancet. 2009 Jul 4;374(9683):86-9 [PMID: 19525005]
  23. Psychol Sci. 2011 Nov;22(11):1359-66 [PMID: 22006061]
  24. Behav Brain Sci. 2018 Jan;41:e124 [PMID: 31064512]
  25. Pers Soc Psychol Rev. 1998;2(3):196-217 [PMID: 15647155]
  26. Psychol Methods. 2012 Mar;17(1):120-8 [PMID: 21787082]
  27. Behav Brain Sci. 2017 Oct 25;41:e120 [PMID: 29065933]
  28. PLoS Med. 2014 Oct 21;11(10):e1001747 [PMID: 25334033]
  29. Psychon Bull Rev. 2009 Aug;16(4):617-40 [PMID: 19648445]
  30. J Pers Soc Psychol. 2011 Mar;100(3):426-32 [PMID: 21280965]
  31. R Soc Open Sci. 2016 Jan 13;3(1):150547 [PMID: 26909182]
  32. Nat Hum Behav. 2018 Jan;2(1):6-10 [PMID: 30980045]
  33. Psychon Bull Rev. 2007 Oct;14(5):779-804 [PMID: 18087943]

Word Cloud

Created with Highcharts 10.0.0replicationpublicationcrisisfindingsstudiesscientificcommunityregimereplicatedreplicabilitycansupportrobustefficientsciencebroadagreementpsychologyfacingEvenseeminglywell-establishedfailedreplicateNumerouscausesidentifiedunderpoweredbiasimprecisetheoriesinadequatestatisticalproceduresreallessclearresolvedexaminepotentialsolutionsmodelingvariousdifferentregimesoneguardsubsequentfailuresalternativeindividualpublishedattractcommunity'sinterestfindpotentiallynon-replicableminimizescostmaximizesefficiencyknowledgegainvarietyassumptionsProvidedproperlymanagedsuggestlowLow

Similar Articles

Cited By