Effective study selection using text mining or a single-screening approach: a study protocol.

Siw Waffenschmidt, Elke Hausner, Wiebke Sieben, Thomas Jaschinski, Marco Knelangen, Inga Overesch
Author Information
  1. Siw Waffenschmidt: Institute for Quality and Efficiency in Health Care, Cologne, Germany. siw.waffenschmidt@iqwig.de. ORCID
  2. Elke Hausner: Institute for Quality and Efficiency in Health Care, Cologne, Germany.
  3. Wiebke Sieben: Institute for Quality and Efficiency in Health Care, Cologne, Germany.
  4. Thomas Jaschinski: Institute for Quality and Efficiency in Health Care, Cologne, Germany.
  5. Marco Knelangen: Institute for Quality and Efficiency in Health Care, Cologne, Germany.
  6. Inga Overesch: Institute for Quality and Efficiency in Health Care, Cologne, Germany.

Abstract

BACKGROUND: Systematic information retrieval generally requires a two-step selection process for studies, which is conducted by two persons independently of one another (double-screening approach). To increase efficiency, two methods seem promising, which will be tested in the planned study: the use of text mining to prioritize search results as well as the involvement of only one person in the study selection process (single-screening approach). The aim of the present study is to examine the following questions related to the process of study selection: Can the use of the Rayyan or EPPI Reviewer tools to prioritize the results of study selection increase efficiency? How accurately does a single-screening approach identify relevant studies? Which advantages or disadvantages (e.g., shortened screening time or increase in the number of full texts ordered) does a single-screening versus a double-screening approach have?
METHODS: Our study is a prospective analysis of study selection processes based on benefit assessments of drug and non-drug interventions. It consists of two parts: firstly, the evaluation of a single-screening approach based on a sample size calculation (11 study selection processes, including 33 single screenings) and involving different screening tools and, secondly, the evaluation of the conventional double-screening approach based on five conventional study selection processes. In addition, the advantages and disadvantages of the single-screening versus the double-screening approach with regard to the outcomes "number of full texts ordered" and "time required for study selection" are analyzed. The previous work experience of the screeners is considered as a potential effect modifier.
DISCUSSION: No study comparing the features of prioritization tools is currently available. Our study can thus contribute to filling this evidence gap. This study is also the first to investigate a range of questions surrounding the screening process and to include an a priori sample size calculation, thus enabling statistical conclusions. In addition, the impact of missing studies on the conclusion of a benefit assessment is calculated.
SYSTEMATIC REVIEW REGISTRATION: Not applicable.

Keywords

References

  1. Syst Rev. 2016 Aug 17;5(1):140 [PMID: 27535658]
  2. Syst Rev. 2015 Jan 14;4:5 [PMID: 25588314]
  3. Res Synth Methods. 2017 Sep;8(3):275-280 [PMID: 28374510]
  4. Syst Rev. 2016 Dec 5;5(1):210 [PMID: 27919275]
  5. J Clin Epidemiol. 2017 Apr;84:33-36 [PMID: 28108352]
  6. Syst Rev. 2015 Jun 15;4:80 [PMID: 26073974]
  7. BMC Med Res Methodol. 2007 Feb 15;7:10 [PMID: 17302989]
  8. J Clin Epidemiol. 2005 May;58(5):444-9 [PMID: 15845330]
  9. Stat Med. 2002 Jun 15;21(11):1635-40 [PMID: 12111924]

MeSH Term

Data Mining
Humans
Information Storage and Retrieval
Research Design
Systematic Reviews as Topic

Word Cloud

Created with Highcharts 10.0.0studyselectionapproachsingle-screeningprocessdouble-screeningscreeningtwoincreasetoolsprocessesbasedSystematicstudiesoneusetextminingprioritizeresultsquestionsadvantagesdisadvantagesfulltextsversusbenefitevaluationsamplesizecalculationconventionaladditionprioritizationthusBACKGROUND:informationretrievalgenerallyrequirestwo-stepconductedpersonsindependentlyanotherefficiencymethodsseempromisingwilltestedplannedstudy:searchwellinvolvementpersonaimpresentexaminefollowingrelatedselection:CanRayyanEPPIReviewerefficiency?accuratelyidentifyrelevantstudies?egshortenedtimenumberorderedhave?METHODS:prospectiveanalysisassessmentsdrugnon-druginterventionsconsistsparts:firstly11including33singlescreeningsinvolvingdifferentsecondlyfiveregardoutcomes"numberordered""timerequiredselection"analyzedpreviousworkexperiencescreenersconsideredpotentialeffectmodifierDISCUSSION:comparingfeaturescurrentlyavailablecancontributefillingevidencegapalsofirstinvestigaterangesurroundingincludepriorienablingstatisticalconclusionsimpactmissingconclusionassessmentcalculatedSYSTEMATICREVIEWREGISTRATION:applicableEffectiveusingapproach:protocolCitationScreeningSemi-automationreviews

Similar Articles

Cited By (3)