Errors to avoid when searching for studies for systematic reviews: A guide for nurse researchers.

Simon Briscoe
Author Information
  1. Simon Briscoe: University of Exeter Medical School, University of Exeter, Exeter, UK. ORCID

Abstract

Systematic reviews aim to provide reliable answers to research questions by identifying and synthesising the available evidence using rigorous methods. This makes systematic reviews a cornerstone of evidence-based practice in healthcare settings. However, despite the avowed aim and importance of systematic reviews, studies have shown that they often include serious flaws, including in the search for studies. In this article, some commonly seen errors in systematic review search strategies are described with the intention of alerting nurse researchers who are planning a systematic review to what should be avoided. These include errors relating to bibliographic databases and supplementary searches, including database selection, free-text searching, controlled vocabulary and structural errors. The paper is framed within the context of older people nursing but has relevance to nurse researchers more widely.

Keywords

References

  1. Abbott, R., Bethel, A., Rogers, M., Whear, R., Orr, N., Shaw, L., Stein, K., & Thompson Coon, J. (2022). Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: A meta-research study. BMJ Evidence Based Medicine, 27(3), 169-177. https://doi.org/10.1136/bmjebm-2021-111710
  2. Abdi, S., Spann, A., Borilovic, J., de Witte, L., & Hawley, M. (2019). Understanding the care and support needs of older people: A scoping review and categorisation using the WHO international classification of functioning, disability and health framework (ICF). BMC Geriatrics, 19(1), 195. https://doi.org/10.1186/s12877-019-1189-9
  3. Aromataris, E., & Munn, Z. (Eds.). (2020). JBI manual for evidence synthesis. JBI.
  4. Aromataris, E., & Riitano, D. (2014). Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. The American Journal of Nursing, 114(5), 49-56. https://doi.org/10.1097/01.NAJ.0000446779.99522.f6
  5. Bethel, A. C., Rogers, M., & Abbott, R. (2021). Use of a search summary table to improve systematic review search methods, results, and efficiency. Journal of the Medical Library Association, 109(1), 97-106. https://doi.org/10.5195/jmla.2021.809
  6. Booth, A. (2016). Searching for qualitative research for inclusion in systematic reviews: A structured methodological review. Systematic Reviews, 5, 74. https://doi.org/10.1186/s13643-016-0249-x
  7. Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review. SAGE Publications.
  8. Briscoe, S., Bethel, A., & Rogers, M. (2020). Conduct and reporting of citation searching in Cochrane systematic reviews: A cross-sectional study. Research Synthesis Methods, 11(2), 169-180. https://doi.org/10.1002/jrsm.1355
  9. Briscoe, S., & Cooper, C. (2014). The British nursing index and CINAHL: A comparison of journal title coverage and the implications for information professionals. Health Information and Libraries Journal, 31(3), 195-203. https://doi.org/10.1111/hir.12069
  10. Briscoe, S., Nunns, M., & Shaw, L. (2020). How do Cochrane authors conduct web searching to identify studies? Findings from a cross-sectional sample of Cochrane reviews. Health Information and Libraries Journal, 37(4), 293-318. https://doi.org/10.1111/hir.12313
  11. Centre for Reviews and Dissemination. (2008). Systematic reviews: CRD's guidance for undertaking reviews in health care. University of York.
  12. Cooper, C., Court, R., Kotas, E., & Schauberger, U. (2021). A technical review of three clinical trials register resources indicates where improvements to the search interfaces are needed. Research Synthesis Methods, 12(3), 384-393. https://doi.org/10.1002/jrsm.1477
  13. Ford, A. C., Guyatt, G. H., Talley, N. J., & Moayyedi, P. (2010). Errors in the conduct of systematic reviews of pharmacological interventions for irritable bowel syndrome. The American Journal of Gastroenterology, 105(2), 280-288. https://doi.org/10.1038/ajg.2009.658
  14. Franco, J. V. A., Garrote, V. L., Escobar Liquitay, C. M., & Vietto, V. (2018). Identification of problems in search strategies in Cochrane reviews. Research Synthesis Methods, 9(3), 408-416. https://doi.org/10.1002/jrsm.1302
  15. Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1, 28. https://doi.org/10.1186/2046-4053-1-28
  16. Gusenbauer, M. (2021). The age of abundant scholarly information and its synthesis-A time when ‘just google it’ is no longer enough. Research Synthesis Methods, 12(6), 684-691. https://doi.org/10.1002/jrsm.1520
  17. Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google scholar, PubMed, and 26 other resources. Research Synthesis Methods, 11(2), 181-217. https://doi.org/10.1002/jrsm.1378
  18. Haddaway, N. R., Bethel, A., Dicks, L. V., Koricheva, J., Macura, B., Petrokofsky, G., Pullin, A. S., Savilaakso, S., & Stewart, G. B. (2020). Eight problems with literature reviews and how to fix them. Nature Ecology and Evolution, 4(12), 1582-1589. https://doi.org/10.1038/s41559-020-01295-x
  19. Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (2019). Cochrane handbook for systematic reviews of interventions (2nd ed.). John Wiley & Sons https://books.google.co.uk/books?id=G1SyDwAAQBAJ
  20. Institute of Medicine Committee on Standards for Systematic Reviews of Comparative Effectiveness, R. (2011). In J. Eden, L. Levit, A. Berg, & S. Morton (Eds.), Finding What Works in Health Care: Standards for systematic reviews. National Academies Press (US). https://doi.org/10.17226/13059
  21. Jones, A. P., Remmington, T., Williamson, P. R., Ashby, D., & Smyth, R. L. (2005). High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. Journal of Clinical Epidemiology, 58(7), 741-742. https://doi.org/10.1016/j.jclinepi.2004.11.024
  22. Koffel, J. B., & Rethlefsen, M. L. (2016). Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: A cross-sectional study. PLoS One, 11(9), e0163309. https://doi.org/10.1371/journal.pone.0163309
  23. Lam, M. T., & McDiarmid, M. (2016). Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association, 104(4), 284-289. https://doi.org/10.3163/1536-5050.104.4.006
  24. Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, M.-I., Noel-Storr, A., Rader, T., Shokraneh, F., Thomas, J., & Wieland, L. (2019). Chapter 4: Searching for and selecting studies. In J. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. Page, & V. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (2nd ed., pp. 67-108). John Wiley & Sons.
  25. Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, M.-I., Noel-Storr, A., Rader, T., Shokraneh, F., Thomas, J., & Wieland, L. (2022). Technical supplement to chapter 4: Searching for and selecting studies. In J. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. Page, & V. Welch (Eds.), Cochrane handbook for systematic reviews of interventions. Version 6.3 (updated February 2022). Cochrane.
  26. McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016). PRESS peer review of electronic search strategies: 2015 guideline Statement. Journal of Clinical Epidemiology, 75, 40-46. https://doi.org/10.1016/j.jclinepi.2016.01.021
  27. National Institutes of Health Library. (n.d.). Systematic Reviews Service. Retrieved September 12 from https://www.nihlibrary.nih.gov/services/systematic-review-service
  28. National Library of Medicine. (n.d.). Medical Subject Headings (MeSH®) in MEDLINE®/PubMed®: A Tutorial. Retrieved September 12 from https://www.nlm.nih.gov/bsd/disted/meshtutorial/meshtreestructures/04.html
  29. Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617-626. https://doi.org/10.1016/j.jclinepi.2014.11.025
  30. Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. B. (2021). PRISMA-S: An extension to the PRISMA Statement for reporting literature searches in systematic reviews. Systematic Reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z
  31. Ruszkowski, J., Majkutewicz, K., Rybka, E., Kutek, M., Dębska-Ślizień, A., & Witkowski, J. M. (2021). The methodological quality and clinical applicability of meta-analyses on probiotics in 2020: A cross-sectional study. Biomedicine and Pharmacotherapy, 142, 112044. https://doi.org/10.1016/j.biopha.2021.112044
  32. Salvador-Oliván, J. A., Marco-Cuenca, G., & Arquero-Avilés, R. (2019). Errors in search strategies used in systematic reviews and their effects on information retrieval. Journal of the Medical Library Association, 107(2), 210-221. https://doi.org/10.5195/jmla.2019.567
  33. Sampson, M., & McGowan, J. (2006). Errors in search strategies were identified by type and frequency. Journal of Clinical Epidemiology, 59(10), 1057-1063. https://doi.org/10.1016/j.jclinepi.2006.01.007
  34. World Health Organisation. (2022). UN Decade of Healthy Ageing 2021-2030. Retrieved December 8 from https://www.who.int/initiatives/decade-of-healthy-ageing

MeSH Term

Humans
Aged
Systematic Reviews as Topic
Information Storage and Retrieval
Databases, Bibliographic
Evidence-Based Practice
Delivery of Health Care

Word Cloud

Created with Highcharts 10.0.0systematicreviewsstudieserrorsnurseresearchersaimincludeincludingsearchreviewsearchingnursingSystematicprovidereliableanswersresearchquestionsidentifyingsynthesisingavailableevidenceusingrigorousmethodsmakescornerstoneevidence-basedpracticehealthcaresettingsHoweverdespiteavowedimportanceshownoftenseriousflawsarticlecommonlyseenstrategiesdescribedintentionalertingplanningavoidedrelatingbibliographicdatabasessupplementarysearchesdatabaseselectionfree-textcontrolledvocabularystructuralpaperframedwithincontextolderpeoplerelevancewidelyErrorsavoidreviews:guide

Similar Articles

Cited By

No available data.