A Priori Justification for Effect Measures in Single-Case Experimental Designs.

Rumen Manolov, Mariola Moeyaert, Joelle E Fingerhut
Author Information
  1. Rumen Manolov: Department de Psicologia Social i Psicologia Quantitativa, Universitat de Barcelona, Passeig de la Vall d´Hebron 171, 08035 Barcelona, Spain. ORCID
  2. Mariola Moeyaert: State University of New York at Albany, Albany, NY USA.
  3. Joelle E Fingerhut: State University of New York at Albany, Albany, NY USA.

Abstract

Due to the complex nature of single-case experimental design data, numerous effect measures are available to quantify and evaluate the effectiveness of an intervention. An inappropriate choice of the effect measure can result in a misrepresentation of the intervention effectiveness and this can have far-reaching implications for theory, practice, and policymaking. As guidelines for reporting appropriate justification for selecting an effect measure are missing, the first aim is to identify the relevant dimensions for effect measure selection and justification prior to data gathering. The second aim is to use these dimensions to construct a user-friendly flowchart or decision tree guiding applied researchers in this process. The use of the flowchart is illustrated in the context of a preregistered protocol. This is the first study that attempts to propose reporting guidelines to justify the effect measure choice, before collecting the data, to avoid selective reporting of the largest quantifications of an effect. A proper justification, less prone to confirmation bias, and transparent and explicit reporting can enhance the credibility of the single-case design study findings.

Keywords

References

  1. Multivariate Behav Res. 2015;50(3):365-80 [PMID: 26610035]
  2. Behav Modif. 2015 Nov;39(6):835-69 [PMID: 26358925]
  3. J Sch Psychol. 2017 Aug;63:13-34 [PMID: 28633936]
  4. Dev Neurorehabil. 2020 Feb;23(2):73-105 [PMID: 31411523]
  5. Behav Modif. 2019 May 12;:145445519847627 [PMID: 31081350]
  6. Behav Res Methods. 2019 Dec;51(6):2847-2869 [PMID: 30484219]
  7. Perspect Behav Sci. 2020 May 26;43(2):285-301 [PMID: 32647783]
  8. Behav Modif. 2019 Apr 9;:145445519839213 [PMID: 30961351]
  9. J Sch Psychol. 2013 Apr;51(2):201-15 [PMID: 23481085]
  10. Res Dev Disabil. 2019 Feb;85:31-41 [PMID: 30453270]
  11. Behav Res Methods. 2020 Feb;52(1):131-150 [PMID: 30805863]
  12. Behav Ther. 2012 Sep;43(3):679-85 [PMID: 22697454]
  13. Psychol Methods. 2014 Dec;19(4):493-510 [PMID: 24933294]
  14. Behav Anal. 1999 Fall;22(2):109-16 [PMID: 22478328]
  15. J Sch Psychol. 2014 Apr;52(2):243-8 [PMID: 24606979]
  16. Psychol Methods. 2017 Dec;22(4):760-778 [PMID: 28358542]
  17. Behav Anal Pract. 2019 Jan 17;13(1):247-252 [PMID: 32231987]
  18. Behav Ther. 2018 Nov;49(6):981-994 [PMID: 30316495]
  19. Psychol Methods. 2012 Dec;17(4):510-50 [PMID: 22845874]
  20. Int J Eat Disord. 2018 Jul;51(7):617-628 [PMID: 29846014]
  21. Am J Intellect Dev Disabil. 2016 May;121(3):169-93 [PMID: 27119210]
  22. Ann Phys Rehabil Med. 2018 May;61(3):164-179 [PMID: 29253607]
  23. Clin J Pain. 2005 Jan-Feb;21(1):56-68; discussion 69-72 [PMID: 15599132]
  24. Behav Res Methods. 2017 Feb;49(1):363-381 [PMID: 26927003]
  25. Am Psychol. 2008 Feb-Mar;63(2):77-95 [PMID: 18284277]
  26. Psychol Methods. 2019 Apr;24(2):217-235 [PMID: 29911874]
  27. Behav Modif. 2004 Mar;28(2):234-46 [PMID: 14997950]
  28. Behav Modif. 2016 Nov;40(6):852-873 [PMID: 27103183]
  29. BMJ. 2020 Feb 27;368:m122 [PMID: 32107202]
  30. Behav Res Methods. 2019 Dec;51(6):2477-2497 [PMID: 30105444]
  31. Behav Modif. 2010 May;34(3):195-218 [PMID: 20234005]
  32. Behav Modif. 2011 Jul;35(4):303-22 [PMID: 21411481]
  33. Behav Res Ther. 2019 Aug;119:103414 [PMID: 31203150]
  34. Res Dev Disabil. 2018 Aug;79:77-87 [PMID: 29289405]
  35. Behav Modif. 2019 Jul;43(4):544-563 [PMID: 29785857]
  36. J Speech Lang Hear Res. 2017 Jun 10;60(6):1455-1466 [PMID: 28595354]
  37. Neuropsychol Rehabil. 2014;24(3-4):554-71 [PMID: 24533782]
  38. Nature. 2015 Oct 8;526(7572):182-5 [PMID: 26450039]
  39. Behav Modif. 2006 Sep;30(5):598-617 [PMID: 16894232]
  40. Perspect Behav Sci. 2019 Mar 11;42(1):1-11 [PMID: 31976418]
  41. Behav Modif. 2014 Jul;38(4):477-96 [PMID: 24235178]
  42. Psychol Methods. 2010 Jun;15(2):124-44 [PMID: 20515235]
  43. J Speech Hear Disord. 1986 Aug;51(3):214-25 [PMID: 3525988]
  44. Behav Modif. 2006 Sep;30(5):531-63 [PMID: 16894229]
  45. Res Dev Disabil. 2018 Aug;79:3-9 [PMID: 29655508]
  46. Res Synth Methods. 2013 Dec;4(4):324-41 [PMID: 26053946]
  47. J Exp Anal Behav. 2019 Mar;111(2):309-328 [PMID: 30706944]
  48. Psychol Methods. 2013 Sep;18(3):385-405 [PMID: 23834421]
  49. Behav Res Methods. 2020 Jun;52(3):1355-1370 [PMID: 31898296]
  50. J Sch Psychol. 2011 Jun;49(3):301-21 [PMID: 21640246]
  51. Behav Ther. 2011 Jun;42(2):284-99 [PMID: 21496513]
  52. J Sch Psychol. 2014 Apr;52(2):213-30 [PMID: 24606976]
  53. J Sch Psychol. 2014 Apr;52(2):149-78 [PMID: 24606973]
  54. Behav Modif. 2017 Jul;41(4):427-467 [PMID: 27831527]
  55. Am J Intellect Dev Disabil. 2019 Jan;124(1):35-56 [PMID: 30715924]
  56. Behav Modif. 2018 Sep;42(5):684-706 [PMID: 28817949]
  57. Neuropsychol Rehabil. 2014;24(3-4):634-60 [PMID: 24779416]
  58. Aphasiology. 2019;33(1):1-30 [PMID: 33012945]
  59. Behav Res Methods. 2019 Dec;51(6):2454-2476 [PMID: 30022457]
  60. Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606 [PMID: 29531091]
  61. J Neurol Phys Ther. 2017 Jul;41(3):187-197 [PMID: 28628553]
  62. J Abnorm Psychol. 2019 Aug;128(6):517-527 [PMID: 31368731]
  63. Behav Modif. 2013 May;37(3):378-90 [PMID: 23408384]
  64. Behav Modif. 2014 Sep;38(5):665-704 [PMID: 24902590]
  65. J Sch Psychol. 2014 Apr;52(2):123-47 [PMID: 24606972]
  66. Neuropsychol Rehabil. 2014;24(3-4):445-63 [PMID: 23883189]
  67. Psychol Sci. 2011 Nov;22(11):1359-66 [PMID: 22006061]
  68. Behav Res Ther. 2019 Jun;117:3-17 [PMID: 30527785]
  69. J Sch Psychol. 2012 Oct;50(5):599-624 [PMID: 23040759]
  70. Perspect Behav Sci. 2018 May 17;42(1):153-162 [PMID: 31976426]
  71. Perspect Behav Sci. 2018 Nov 14;42(1):13-31 [PMID: 31976419]
  72. Behav Modif. 2019 Aug 2;:145445519864264 [PMID: 31375029]
  73. Behav Res Methods. 2020 Feb;52(1):177-192 [PMID: 30972557]
  74. J Appl Behav Anal. 2017 Oct;50(4):701-716 [PMID: 28887866]
  75. Behav Res Ther. 1994 Sep;32(7):783-6 [PMID: 7980365]
  76. Dev Neurorehabil. 2018 May;21(4):266-278 [PMID: 26809945]
  77. Sch Psychol Q. 2018 Dec;33(4):590-603 [PMID: 30284889]
  78. Adm Policy Ment Health. 2019 Jul;46(4):445-457 [PMID: 30783903]
  79. Behav Ther. 2017 Jan;48(1):97-114 [PMID: 28077224]
  80. Behav Ther. 2009 Dec;40(4):357-67 [PMID: 19892081]
  81. Behav Modif. 2015 Jul;39(4):510-41 [PMID: 25878161]
  82. Neuropsychol Rehabil. 2014;24(3-4):507-27 [PMID: 23865938]
  83. Behav Modif. 2017 Mar;41(2):179-228 [PMID: 27553741]
  84. J Sch Psychol. 2014 Apr;52(2):191-211 [PMID: 24606975]
  85. Contemp Clin Trials Commun. 2019 Sep 27;16:100455 [PMID: 31650075]
  86. Dev Neurorehabil. 2018 Jul;21(5):290-311 [PMID: 27367902]
  87. Behav Modif. 2020 Jan;44(1):27-48 [PMID: 30079759]
  88. J Consult Clin Psychol. 1981 Apr;49(2):193-211 [PMID: 7217485]
  89. J Sch Psychol. 2018 Jun;68:99-112 [PMID: 29861034]
  90. J Sch Psychol. 2016 Jun;56:133-42 [PMID: 27268573]
  91. Res Dev Disabil. 2018 Aug;79:10-18 [PMID: 29960830]
  92. Multivariate Behav Res. 2019 Sep-Oct;54(5):666-689 [PMID: 30857444]
  93. Psychol Sci. 2015 Dec;26(12):1827-32 [PMID: 26553013]
  94. Behav Modif. 2020 Jul;44(4):518-551 [PMID: 30931585]
  95. Dev Neurorehabil. 2018 May;21(4):253-265 [PMID: 26864247]
  96. Behav Res Ther. 2019 Jun;117:18-27 [PMID: 30670306]
  97. Behav Res Methods Instrum Comput. 2003 Feb;35(1):1-10 [PMID: 12723775]
  98. Psychol Methods. 2018 Sep;23(3):480-504 [PMID: 28301199]
  99. Neuropsychol Rehabil. 2021 Apr;31(3):369-391 [PMID: 31793383]
  100. Behav Res Methods. 2009 May;41(2):372-84 [PMID: 19363177]
  101. Behav Res Methods. 2020 Apr;52(2):654-666 [PMID: 31270794]
  102. Res Dev Disabil. 2018 Aug;79:88-96 [PMID: 29807692]
  103. Multivariate Behav Res. 2018 Jul-Aug;53(4):574-593 [PMID: 29757002]
  104. Res Synth Methods. 2012 Sep;3(3):224-39 [PMID: 26062165]

Word Cloud

Created with Highcharts 10.0.0effectreportingmeasuresingle-casedesigndatacanjustificationexperimentaleffectivenessinterventionchoiceguidelinesfirstaimdimensionsuseflowchartstudyDuecomplexnaturenumerousmeasuresavailablequantifyevaluateinappropriateresultmisrepresentationfar-reachingimplicationstheorypracticepolicymakingappropriateselectingmissingidentifyrelevantselectionpriorgatheringsecondconstructuser-friendlydecisiontreeguidingappliedresearchersprocessillustratedcontextpreregisteredprotocolattemptsproposejustifycollectingavoidselectivelargestquantificationsproperlessproneconfirmationbiastransparentexplicitenhancecredibilityfindingsPrioriJustificationEffectMeasuresSingle-CaseExperimentalDesignsquantitativemethodsstandardsscientificrigorstatisticalanalysis

Similar Articles

Cited By