OpenLB
Open Library of Bioscience
Advanced Search
Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing
Sep, 2024;56 (5): 705-717.

Particularity, Engagement, Actionable Inferences, Reflexivity, and Legitimation tool for rigor in mixed methods implementation research.

Ahtisham Younas, Sergi Fàbregues
Author Information
  1. Ahtisham Younas: Faculty of Nursing, Memorial University of Newfoundland, St. John's, Newfoundland, Canada. ORCID
  2. Sergi Fàbregues: Department of Psychology and Education, Universitat Oberta de Catalunya, Barcelona, Spain.
PMID: 38691344 DOI: 10.1111/jnu.12977

Abstract

BACKGROUND: Implementation science helps generate approaches to expedite the uptake of evidence in practice. Mixed methods are commonly used in implementation research because they allow researchers to integrate distinct qualitative and quantitative methods and data sets to unravel the implementation process and context and design contextual tools for optimizing the implementation. To date, there has been limited discussion on how to ensure rigor in mixed methods implementation research.
PURPOSE: To present Particularity, Engagement, Actionable Inferences, Reflexivity, and Legitimation (PEARL) as a practical tool for understanding various components of rigor in mixed methods implementation research.
DATA SOURCES: This methodological discussion is based on a nurse-led mixed methods implementation study. The PEARL tool was developed based on an interpretive, critical reflection, and purposive reading of selected literature sources drawn from the researchers' knowledge, experiences of designing and conducting mixed methods implementation research, and published methodological papers about mixed methods, implementation science, and research rigor.
CONCLUSION: An exemplar exploratory sequential mixed methods study in nursing is provided to illustrate the application of the PEARL tool. The proposed tool can be a useful and innovative tool for researchers and students intending to use mixed methods in implementation research. The tool offers a straightforward approach to learning the key rigor components of mixed methods implementation research for application in designing and conducting implementation research using mixed methods.
CLINICAL RELEVANCE: Rigorous implementation research is critical for effective uptake of innovations and evidence-based knowledge into practice and policymaking. The proposed tool can be used as the means to establish rigor in mixed methods implementation research in nursing and health sciences.

Keywords

implementation research implementation science mixed methods research methodology rigor

References

  1. Ashcraft, L. E., Cabrera, K. I., Lane‐Fall, M. B., & South, E. C. (2024). Leveraging implementation science to advance environmental justice research and achieve health equity through neighborhood and policy interventions. Annual Review of Public Health, 45. 24.1–24.20. https://doi.org/10.1146/annurev‐publhealth‐060222‐033003
  2. Atkins, L., Francis, J., Islam, R., O'Connor, D., Patey, A., Ivers, N., Foy, R., Duncan, E. M., Colquhoun, H., Grimshaw, J. M., Lawton, R., & Michie, S. (2017). A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implementation Science, 12(1), 77. https://doi.org/10.1186/s13012‐017‐0605‐9
  3. Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015). An introduction to implementation science for the non‐specialist. BMC Psychology, 3(1), 32. https://doi.org/10.1186/s40359‐015‐0089‐9
  4. Bauer, M. S., & Kirchner, J. (2020). Implementation science: What is it and why should I care? Psychiatry Research, 283, 112376. https://doi.org/10.1016/j.psychres.2019.04.025
  5. Birken, S. A., Powell, B. J., Shea, C. M., Haines, E. R., Alexis Kirk, M., Leeman, J., Rohweder, C., Damschroder, L., & Presseau, J. (2017). Criteria for selecting implementation science theories and frameworks: Results from an international survey. Implementation Science, 12(1), 124. https://doi.org/10.1186/s13012‐017‐0656‐y
  6. Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
  7. Braun, V., & Clarke, V. (2021). Conceptual and design thinking for thematic analysis. Qualitative Psychology, 9(1), 3–26. https://doi.org/10.1037/qup0000196
  8. Brown, C. H., Curran, G., Palinkas, L. A., Aarons, G. A., Wells, K. B., Jones, L., Collins, L. M., Duan, N., Mittman, B. S., Wallace, A., Tabak, R. G., Ducharme, L., Chambers, D. A., Neta, G., Wiley, T., Landsverk, J., Cheung, K., & Cruden, G. (2017). An overview of research and evaluation designs for dissemination and implementation. Annual Review of Public Health, 38, 1–22. https://doi.org/10.1146/annurev‐publhealth‐031816‐044215
  9. Brown, M., Rosenthal, M., & Yeh, D. D. (2021). Implementation science and nutrition: From research to practice. Nutrition in Clinical Practice, 36(3), 586–597. https://doi.org/10.1002/ncp.10677
  10. Brownson, R. C., Shelton, R. C., Geng, E. H., & Glasgow, R. E. (2022). Revisiting concepts of evidence in implementation science. Implementation Science, 17(1), 26. https://doi.org/10.1186/s13012‐022‐01201‐y
  11. Cane, J., O'Connor, D., & Michie, S. (2012). Validation of the theoretical domains framework for use in behaviour change and implementation research. Implementation Science, 7, 37. https://doi.org/10.1186/1748‐5908‐7‐37
  12. Cassidy, C. E., Harrison, M. B., Godfrey, C., Nincic, V., Khan, P. A., Oakley, P., Ross‐White, A., Grantmyre, H., & Graham, I. D. (2021). Use and effects of implementation strategies for practice guidelines in nursing: A systematic review. Implementation Science, 16(1), 1–29. https://doi.org/10.1186/s13012‐021‐01165‐5
  13. Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. The Lancet, 374(9683), 86–89. https://doi.org/10.1016/S0140‐6736(09)60329‐9
  14. Collins, K. M. T., Onwuegbuzie, A. J., & Johnson, R. B. (2012). Securing a place at the table. American Behavioral Scientist, 56(6), 849–865. https://doi.org/10.1177/0002764211433799
  15. Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.
  16. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness‐implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226. https://doi.org/10.1097/MLR.0b013e3182408812
  17. Curran, G. M., Landes, S. J., McBain, S. A., Pyne, J. M., Smith, J. D., Fernandez, M. E., Chambers, D. A., & Mittman, B. S. (2022). Reflections on 10 years of effectiveness‐implementation hybrid studies. Frontiers in Health Services, 2, 1053496. https://doi.org/10.3389/frhs.2022.1053496
  18. Dopp, A. R., Mundey, P., Beasley, L. O., Silovsky, J. F., & Eisenberg, D. (2019). Mixed‐method approaches to strengthen economic evaluations in implementation research. Implementation Science, 14, 1–9. https://doi.org/10.1186/s13012‐018‐0850‐6
  19. Fetters, M. D. (2019). The mixed methods research workbook: Activities for designing, implementing, and publishing projects. SAGE.
  20. Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs‐principles and practices. Health Services Research, 48(6 Pt 2), 2134–2156. https://doi.org/10.1111/1475‐6773.12117
  21. Finlay, L. (2002). “outing” the researcher: The provenance, process, and practice of reflexivity. Qualitative Health Research, 12(4), 531–545. https://doi.org/10.1177/104973202129120052
  22. Finney Rutten, L. J., Ridgeway, J. L., & Griffin, J. M. (2024). Advancing translation of clinical research into practice and population health impact through implementation science. Mayo Clinic Proceedings, 99(4), 665–676. https://doi.org/10.1016/j.mayocp.2023.02.005
  23. Flemming, K. (2007). The knowledge base for evidence‐based nursing: A role for mixed methods research? Advances in Nursing Science, 30(1), 41–51. https://doi.org/10.1097/00012272‐200701000‐00005
  24. Geng, E. H., Peiris, D., & Kruk, M. E. (2017). Implementation science: Relevance in the real world without sacrificing rigor. PLoS Medicine, 14(4), e1002288. https://doi.org/10.1371/journal.pmed.1002288
  25. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE‐AIM framework. American Journal of Public Health, 89(9), 1322–1327. https://doi.org/10.2105/ajph.89.9.1322
  26. Goodman, M. S., & Sanders Thompson, V. L. (2017). The science of stakeholder engagement in research: Classification, implementation, and evaluation. Translational Behavioral Medicine, 7(3), 486–491. https://doi.org/10.1007/s13142‐017‐0495‐z
  27. Goorts, K., Dizon, J., & Milanese, S. (2021). The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: A systematic review. BMC Health Services Research, 21(1), 1–11. https://doi.org/10.1186/s12913‐021‐06190‐0
  28. Hamilton, A. B., & Finley, E. P. (2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, 112516. https://doi.org/10.1016/j.psychres.2019.112516
  29. Harrison, M. B., & Graham, I. D. (2021). Knowledge translation in nursing and healthcare: A roadmap to evidence‐informed practice. John Wiley & Sons.
  30. Jamieson, M. K., Pownall, M., & Govaart, G. H. (2022). Reflexivity in quantitative research: A rationale and beginner's guide. Social and Personality Psychology Compass, 17, e12735. https://psyarxiv.com/xvrhm
  31. Johnson, R. B., & Christensen, L. (2017). Validity of research results in quantitative, qualitative and mixed research. In R. B. Johnson & L. Christensen (Eds.), Educational research. Quantitative, qualitative, and mixed approaches (6th ed., pp. 306–310). SAGE.
  32. Khan, S., Chambers, D., & Neta, G. (2021). Revisiting time to translation: Implementation of evidence‐based practices (EBPs) in cancer control. Cancer Causes & Control, 32(3), 221–230. https://doi.org/10.1007/s10552‐020‐01376‐z
  33. Kilbourne, A., Chinman, M., Rogal, S., & Almirall, D. (2023). Adaptive designs in implementation science and practice: Their promise and the need for greater understanding and improved communication. Annual Review of Public Health, 45, 4.1–4.20. https://doi.org/10.1146/annurev‐publhealth‐060222‐014438
  34. Kilbourne, A. M., Glasgow, R. E., & Chambers, D. A. (2020). What can implementation science do for you? Key success stories from the field. Journal of General Internal Medicine, 35(S2), 783–787. https://doi.org/10.1007/s11606‐020‐06174‐6
  35. Landes, S. J., McBain, S. A., & Curran, G. M. (2019). An introduction to effectiveness‐implementation hybrid designs. Psychiatry Research, 280, 112513. https://doi.org/10.1016/j.psychres.2019.112513
  36. Leko, M. M., Hitchcock, J. H., Love, H. R., Houchins, D. E., & Conroy, M. A. (2023). Quality indicators for mixed‐methods research in special education. Exceptional Children, 89(4), 432–448. https://doi.org/10.1177/00144029221141031
  37. Lengnick‐Hall, R., Williams, N. J., Ehrhart, M. G., Willging, C. E., Bunger, A. C., Beidas, R. S., & Aarons, G. A. (2023). Eight characteristics of rigorous multilevel implementation research: A step‐by‐step guide. Implementation Science, 18(1), 52. https://doi.org/10.1186/s13012‐023‐01302‐2
  38. Manning, E., & Gagnon, M. (2017). The complex patient: A concept clarification. Nursing & Health Sciences, 19(1), 13–21. https://doi.org/10.1111/nhs.12320
  39. Matović, N., & Ovesni, K. (2023). Interaction of quantitative and qualitative methodology in mixed methods research: Integration and/or combination. International Journal of Social Research Methodology, 26(1), 51–65. https://doi.org/10.1080/13645579.2021.1964857
  40. Mertens, D. M. (2023) Mixed methods research: Research methods. Bloomsbury.
  41. Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013). The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine, 46(1), 81–95. https://doi.org/10.1007/s12160‐013‐9486‐6
  42. Michie, S., van Stralen, M. M., & West, R. (2011). The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science, 6, 42. https://doi.org/10.1186/1748‐5908‐6‐42
  43. Morris, Z. S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: Understanding time lags in translational research. Journal of the Royal Society of Medicine, 104(12), 510–520. https://doi.org/10.1258/jrsm.2011.110180
  44. Nastasi, B. K., & Hitchcock, J. (2016). Mixed methods research and culture‐specific interventions: Program design and evaluation. SAGE.
  45. Nilsen, P. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10, 53. https://doi.org/10.1186/s13012‐015‐0242‐0
  46. Onwuegbuzie, A. J., & Johnson, R. B. (2006). The validity issue in mixed research. Research in the Schools, 13(1), 48–63.
  47. Onwuegbuzie, A. J., Johnson, R. B., & Collins, K. M. (2011). Assessing legitimation in mixed research: A new framework. Quality & Quantity, 45(6), 1253–1271. https://doi.org/10.1007/s11135‐009‐9289‐9
  48. Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health, 38(1), 44–53. https://doi.org/10.1007/s10488‐010‐0314‐z
  49. Palinkas, L. A., & Cooper, B. R. (2017). Mixed methods evaluation in dissemination and implementation science. In R. C. Brownson, G. A. Colditz, & E. K. Proctor (Eds.), Dissemination and implementation research in health: Translating science to practice (2nd ed., pp. 335–354). Oxford University Press. https://doi.org/10.1093/oso/9780190683214.003.0020
  50. Pearson, N., Naylor, P. J., Ashe, M. C., Fernandez, M., Yoong, S. L., & Wolfenden, L. (2020). Guidance for conducting feasibility and pilot studies for implementation trials. Pilot and Feasibility Studies, 6(1), 167. https://doi.org/10.1186/s40814‐020‐00634‐w
  51. Pellecchia, M., Arnold, K. T., Tomczuk, L., & Beidas, R. S. (2023). Engaging Stakeholders. In B. J. Weiner, C. C. Lewis, & K. Sherr (Eds.), Practical implementation science: Moving evidence into action (pp. 133–154). Springer Publishing Company.
  52. Perez, A., Howell Smith, M. C., Babchuk, W. A., & Lynch‐O'Brien, L. I. (2023). Advancing quality standards in mixed methods research: Extending the legitimation typology. Journal of Mixed Methods Research, 17(1), 29–50. https://doi.org/10.1177/15586898221093872
  53. Pérez Jolles, M., Willging, C. E., Stadnick, N. A., Crable, E. L., Lengnick‐Hall, R., Hawkins, J., & Aarons, G. A. (2022). Understanding implementation research collaborations from a co‐creation lens: Recommendations for a path forward. Frontiers in Health Services, 2, 942658. https://doi.org/10.3389/frhs.2022.942658
  54. Peters, D. H., Adam, T., Alonge, O., Agyepong, I. A., & Tran, N. (2013). Implementation research: What it is and how to do it. BMJ, 347, f6753. https://doi.org/10.1136/bmj.f6753
  55. Peters, D. H., Bhuiya, A., & Ghaffar, A. (2017). Engaging stakeholders in implementation research: Lessons from the future health systems research programme experience. Health Research Policy and Systems, 15(Suppl 2), 104. https://doi.org/10.1186/s12961‐017‐0269‐6
  56. Pluye, P., Grad, R. M., Levine, A., & Nicolau, B. (2009). Understanding divergence of quantitative and qualitative data (or results) in mixed methods studies. International Journal of Multiple Research Approaches, 3(1), 58–72. https://doi.org/10.5172/mra.455.3.1.58
  57. Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the expert recommendations for implementing change (ERIC) project. Implementation Science, 10, 21. https://doi.org/10.1186/s13012‐015‐0209‐1
  58. Presseau, J., McCleary, N., Lorencatto, F., Patey, A. M., Grimshaw, J. M., & Francis, J. J. (2019). Action, actor, context, target, time (AACTT): A framework for specifying behaviour. Implementation Science, 14(1), 102. https://doi.org/10.1186/s13012‐019‐0951‐x
  59. Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34. https://doi.org/10.1007/s10488‐008‐0197‐4
  60. Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8(1), 1–11. https://doi.org/10.1186/1748‐5908‐8‐139
  61. Ramanadhan, S., Revette, A. C., Lee, R. M., & Aveling, E. L. (2021). Pragmatic approaches to analyzing qualitative data for implementation science: An introduction. Implementation Science Communications, 2(1), 70. https://doi.org/10.1186/s43058‐021‐00174‐1
  62. Rapport, F., Clay‐Williams, R., Churruca, K., Shih, P., Hogden, A., & Braithwaite, J. (2018). The struggle of translating science into action: Foundational concepts of implementation science. Journal of Evaluation in Clinical Practice, 24(1), 117–126. https://doi.org/10.1111/jep.12741
  63. Russell, G. M., & Kelly, N. H. (2002). Research as interacting dialogic processes: Implications for reflexivity. Qualitative Social Research, 3(3), 18. https://doi.org/10.17169/fqs‐3.3.831
  64. Snell‐Rood, C., Jaramillo, E. T., Hamilton, A. B., Raskin, S. E., Nicosia, F. M., & Willging, C. (2021). Advancing health equity through a theoretically critical implementation science. Translational Behavioral Medicine, 11(8), 1617–1625. https://doi.org/10.1093/tbm/ibab008
  65. Straus, S. E., & Tetroe, J. (Eds.). (2013). Knowledge translation in health care: Moving from evidence to practice. John Wiley and Sons.
  66. Strifler, L., Cardoso, R., McGowan, J., Cogo, E., Nincic, V., Khan, P. A., Scott, A., Ghassemi, M., MacDonald, H., Lai, Y., Treister, V., Tricco, A. C., & Straus, S. E. (2018). Scoping review identifies significant number of knowledge translation theories, models, and frameworks with limited use. Journal of Clinical Epidemiology, 100, 92–102. https://doi.org/10.1016/j.jclinepi.2018.04.008
  67. Tashakkori, A., & Teddlie, C. (2008). Quality of inferences in mixed methods research. In M. M. Bergman (Ed.), Advances in mixed methods research: Theories and applications (pp. 53–65). Sage.
  68. Walker, S., Read, S., & Priest, H. (2013). Use of reflexivity in a mixed‐methods study. Nurse Researcher, 20(3), 38–43. https://doi.org/10.7748/nr2013.01.20.3.38.c9496
  69. Wolfenden, L., Foy, R., Presseau, J., Grimshaw, J. M., Ivers, N. M., Powell, B. J., Taljaard, M., Wiggers, J., Sutherland, R., Nathan, N., Williams, C. M., Kingsland, M., Milat, A., Hodder, R. K., & Yoong, S. L. (2021). Designing and undertaking randomised implementation trials: Guide for researchers. BMJ, 372, m3721. https://doi.org/10.1136/bmj.m3721
  70. Woodward, E. N., & Ball, I. A. (2023). Perspectives on learning to practice reflexivity while engaging communities in implementation science. Frontiers in Health Services, 2, 1070444. https://doi.org/10.3389/frhs.2022.1070444
  71. Younas, A. (2023a). Value of implementation science and hybrid implementation research designs for nursing education research: A discussion paper. Nurse Education in Practice, 70, 103650. https://doi.org/10.1016/j.nepr.2023.103650
  72. Younas, A. (2023b). Uptake of innovations in nursing: The necessity for implementation science. Creative Nursing, 29(2), 177–181. https://doi.org/10.1177/10784535231195426
  73. Younas, A., & Durante, A. (2023). Decision tree for identifying pertinent integration procedures and joint displays in mixed methods research. Journal of Advanced Nursing, 79(7), 2754–2769. https://doi.org/10.1111/jan.15536
  74. Younas, A., Durante, A., & Fàbregues, S. (2023). Understanding the nature of and identifying and formulating “research problems” in mixed methods research. Journal of Mixed Methods Research. https://doi.org/10.1177/15586898231191441
  75. Younas, A., Maddigan, J., Moore, J. E., & Whitehead, D. (2024). Designing and conducting Q methodology in implementation research: A methodological discussion. Global Implementation Research and Applications, 1–14. https://doi.org/10.1007/s43477‐023‐00113‐3
  76. Younas, A., Pedersen, M., & Inayat, S. (2023). Practical strategies to identify and address discordant findings in mixed methods research. Research Methods in Medicine & Health Sciences, 4(1), 24–33.
  77. Younas, A., Porr, C., Maddigan, J., Moore, J., Navarro, P., & Whitehead, D. (2023). Implementation strategies to promote compassionate nursing care of complex patients: An exploratory sequential mixed methods study. Journal of Nursing Scholarship, 55(4), 805–824. https://doi.org/10.1111/jnu.12869

MeSH Term

Humans
Research Design
Nursing Research
Qualitative Research
Implementation Science
Evidence-Based Nursing
Journal Article Review
Article has an altmetric score of 4

Word Cloud

Created with Highcharts 10.0.0implementationmethodsresearchmixedtoolrigorsciencePEARLuptakepracticeusedresearchersdiscussionParticularityEngagementActionableInferencesReflexivityLegitimationcomponentsmethodologicalbasedstudycriticalknowledgedesigningconductingnursingapplicationproposedcanBACKGROUND:ImplementationhelpsgenerateapproachesexpediteevidenceMixedcommonlyallowintegratedistinctqualitativequantitativedatasetsunravelprocesscontextdesigncontextualtoolsoptimizingdatelimitedensurePURPOSE:presentpracticalunderstandingvariousDATASOURCES:nurse-leddevelopedinterpretivereflectionpurposivereadingselectedliteraturesourcesdrawnresearchers'experiencespublishedpapersCONCLUSION:exemplarexploratorysequentialprovidedillustrateusefulinnovativestudentsintendinguseoffersstraightforwardapproachlearningkeyusingCLINICALRELEVANCE:Rigorouseffectiveinnovationsevidence-basedpolicymakingmeansestablishhealthsciencesmethodology

Similar Articles

Cited By