Can editors save peer review from peer reviewers?

Rafael D'Andrea, James P O'Dwyer
Author Information
  1. Rafael D'Andrea: Dept of Plant Biology, University of Illinois, Urbana-Champaign, Illinois, United States of America. ORCID
  2. James P O'Dwyer: Dept of Plant Biology, University of Illinois, Urbana-Champaign, Illinois, United States of America.

Abstract

Peer review is the gold standard for scientific communication, but its ability to guarantee the quality of published research remains difficult to verify. Recent modeling studies suggest that peer review is sensitive to reviewer misbehavior, and it has been claimed that referees who sabotage work they perceive as competition may severely undermine the quality of publications. Here we examine which aspects of suboptimal reviewing practices most strongly impact quality, and test different mitigating strategies that editors may employ to counter them. We find that the biggest hazard to the quality of published literature is not selfish rejection of high-quality manuscripts but indifferent acceptance of low-quality ones. Bypassing or blacklisting bad reviewers and consulting additional reviewers to settle disagreements can reduce but not eliminate the impact. The other editorial strategies we tested do not significantly improve quality, but pairing manuscripts to reviewers unlikely to selfishly reject them and allowing revision of rejected manuscripts minimize rejection of above-average manuscripts. In its current form, peer review offers few incentives for impartial reviewing efforts. Editors can help, but structural changes are more likely to have a stronger impact.

References

  1. Springerplus. 2016 Jun 27;5(1):903 [PMID: 27386349]
  2. JAMA. 2002 Jun 5;287(21):2784-6 [PMID: 12038911]
  3. PLoS One. 2016 Nov 10;11(11):e0166387 [PMID: 27832157]
  4. Scientometrics. 2014;99:663-688 [PMID: 24829514]
  5. BMJ. 2010 Nov 16;341:c6425 [PMID: 21081603]
  6. F1000Res. 2017 Jul 20;6:1151 [PMID: 29188015]
  7. BMJ. 2010 Nov 16;341:c6424 [PMID: 21081602]
  8. Nature. 2017 Jun 14;546(7658):352 [PMID: 28617464]
  9. Front Comput Neurosci. 2012 Oct 17;6:79 [PMID: 23087639]
  10. Nature. 2016 Apr 21;532(7599):306-8 [PMID: 27111616]
  11. PLoS One. 2011 Apr 12;6(4):e18680 [PMID: 21533268]
  12. PLoS Biol. 2007 Apr;5(4):e107 [PMID: 17439298]
  13. Scientometrics. 2017;113(1):651-671 [PMID: 29056795]
  14. Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000016 [PMID: 17443635]
  15. F1000Res. 2017 Apr 27;6:588 [PMID: 28580134]
  16. Brain. 2000 Sep;123 ( Pt 9):1964-9 [PMID: 10960059]
  17. BMJ. 1999 Jan 2;318(7175):23-7 [PMID: 9872878]

MeSH Term

Communication
Editorial Policies
Humans
Peer Review, Research
Publishing
Social Behavior

Word Cloud

Created with Highcharts 10.0.0qualityreviewpeermanuscriptsimpactreviewerspublishedmayreviewingstrategieseditorsrejectioncanPeergoldstandardscientificcommunicationabilityguaranteeresearchremainsdifficultverifyRecentmodelingstudiessuggestsensitivereviewermisbehaviorclaimedrefereessabotageworkperceivecompetitionseverelyunderminepublicationsexamineaspectssuboptimalpracticesstronglytestdifferentmitigatingemploycounterfindbiggesthazardliteratureselfishhigh-qualityindifferentacceptancelow-qualityonesBypassingblacklistingbadconsultingadditionalsettledisagreementsreduceeliminateeditorialtestedsignificantlyimprovepairingunlikelyselfishlyrejectallowingrevisionrejectedminimizeabove-averagecurrentformoffersincentivesimpartialeffortsEditorshelpstructuralchangeslikelystrongerCansavereviewers?

Similar Articles

Cited By