Replication initiatives will not salvage the trustworthiness of psychology.

James C Coyne
Author Information
  1. James C Coyne: Department of Health Psychology, University Medical Center, University of Groningen, P.O. Box 196, 9700 AD, Groningen, The Netherlands. jcoynester@gmail.com.

Abstract

Replication initiatives in psychology continue to gather considerable attention from far outside the field, as well as controversy from within. Some accomplishments of these initiatives are noted, but this article focuses on why they do not provide a general solution for what ails psychology. There are inherent limitations to mass replications ever being conducted in many areas of psychology, both in terms of their practicality and their prospects for improving the science. Unnecessary compromises were built into the ground rules for design and publication of the Open Science Collaboration: Psychology that undermine its effectiveness. Some ground rules could actually be flipped into guidance for how not to conduct replications. Greater adherence to best publication practices, transparency in the design and publishing of research, strengthening of independent post-publication peer review and firmer enforcement of rules about data sharing and declarations of conflict of interest would make many replications unnecessary. Yet, it has been difficult to move beyond simple endorsement of these measures to consistent implementation. Given the strong institutional support for questionable publication practices, progress will depend on effective individual and collective use of social media to expose lapses and demand reform. Some recent incidents highlight the necessity of this.

Keywords

References

  1. PLoS One. 2012;7(8):e40808 [PMID: 22870204]
  2. J Natl Cancer Inst. 2012 Jul 3;104(13):990-1004 [PMID: 22767203]
  3. J Psychosom Res. 2012 Dec;73(6):408-10 [PMID: 23148806]
  4. J Natl Cancer Inst. 2012 Nov 21;104(22):1770; author reply 1770-1 [PMID: 23000656]
  5. BMC Med. 2013;11:11 [PMID: 23324495]
  6. J Psychosom Res. 2013 Jul;75(1):1-17 [PMID: 23751231]
  7. Psychol Sci. 2013 Jul 1;24(7):1123-32 [PMID: 23649562]
  8. Int J Health Geogr. 2013;12:33 [PMID: 23816238]
  9. Proc Natl Acad Sci U S A. 2013 Aug 13;110(33):13684-9 [PMID: 23898182]
  10. Proc Natl Acad Sci U S A. 2013 Nov 5;110(45):E4183 [PMID: 24135004]
  11. BMJ. 2014;348:g1464 [PMID: 24625460]
  12. Acta Psychol (Amst). 2014 May;148:188-94 [PMID: 24589374]
  13. Lancet. 2014 Aug 2;384(9941):400 [PMID: 25088854]
  14. Proc Natl Acad Sci U S A. 2014 Sep 2;111(35):E3581 [PMID: 25157140]
  15. Proc Natl Acad Sci U S A. 2014 Sep 2;111(35):12705-9 [PMID: 25157145]
  16. J Pers Soc Psychol. 2015 Feb;108(2):275-97 [PMID: 25603376]
  17. PLoS One. 2015;10(3):e0121839 [PMID: 25811656]
  18. Psychol Sci. 2015 Jul;26(7):1140-3 [PMID: 26025022]
  19. Psychol Sci. 2015 Jul;26(7):1144-6 [PMID: 26025023]
  20. Science. 2015 Aug 28;349(6251):aac4716 [PMID: 26315443]
  21. Nature. 2016 Feb 4;530(7588):7 [PMID: 26842021]
  22. Lancet Psychiatry. 2016 Feb;3(2):e6-7 [PMID: 26795757]
  23. Lancet Psychiatry. 2016 Feb;3(2):e8-9 [PMID: 26795760]
  24. Science. 2016 Mar 4;351(6277):1037 [PMID: 26941311]
  25. Science. 2016 Mar 4;351(6277):1037 [PMID: 26941312]
  26. J Cardiovasc Risk. 2002 Feb;9(1):41-8 [PMID: 11984216]
  27. J Consult Clin Psychol. 2003 Dec;71(6):1036-48 [PMID: 14622079]
  28. J Clin Epidemiol. 2006 Sep;59(9):964-9 [PMID: 16895820]
  29. Pers Soc Psychol Bull. 2007 Sep;33(9):1251-64 [PMID: 17565050]
  30. Health Psychol. 2010 Mar;29(2):107-16 [PMID: 20230082]
  31. BMC Med. 2010;8:18 [PMID: 20334633]
  32. Lancet. 2011 Mar 5;377(9768):823-36 [PMID: 21334061]
  33. J Psychiatr Res. 2011 May;45(5):626-9 [PMID: 21035130]
  34. Psychosom Med. 2011 Sep;73(7):528-32 [PMID: 21873588]
  35. Psychol Sci. 2011 Nov;22(11):1359-66 [PMID: 22006061]
  36. PLoS One. 2011;6(11):e26828 [PMID: 22073203]
  37. J Psychosom Res. 2012 Dec;73(6):401-7 [PMID: 23148805]

MeSH Term

Humans
Peer Review, Research
Psychology
Reproducibility of Results
Research Design

Word Cloud

Created with Highcharts 10.0.0psychologyinitiativesreplicationsrulespublicationReplicationmanygrounddesignpracticeswillcontinuegatherconsiderableattentionfaroutsidefieldwellcontroversywithinaccomplishmentsnotedarticlefocusesprovidegeneralsolutionailsinherentlimitationsmasseverconductedareastermspracticalityprospectsimprovingscienceUnnecessarycompromisesbuiltOpenScienceCollaboration:PsychologyundermineeffectivenessactuallyflippedguidanceconductGreateradherencebesttransparencypublishingresearchstrengtheningindependentpost-publicationpeerreviewfirmerenforcementdatasharingdeclarationsconflictinterestmakeunnecessaryYetdifficultmovebeyondsimpleendorsementmeasuresconsistentimplementationGivenstronginstitutionalsupportquestionableprogressdependeffectiveindividualcollectiveusesocialmediaexposelapsesdemandreformrecentincidentshighlightnecessitysalvagetrustworthinessPublicationbiasRandomizedcontrolledtrialsReproducibilityp-hacking

Similar Articles

Cited By