Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue.

George W Howe, C Hendricks Brown
Author Information
  1. George W Howe: Department of Psychological and Brain Sciences, George Washington University, 2103 H Street NW, 20052, Washington, DC, USA. ghowe@gwu.edu. ORCID
  2. C Hendricks Brown: Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.

Abstract

The current special issue of Prevention Science indicates that momentum in using individual participant data (IPD) and integrative data analysis (IDA) to combine and synthesize findings in prevention science has accelerated over the past decade. In this commentary, we focus on two general themes involving methods for harmonizing measures and findings of effect heterogeneity. We describe methods for harmonization as retrospective psychometrics, requiring that we attend to the assumptions necessary for accurate measurement, but adjust our methods given the constraints of working with existing datasets that often involve different measures in different studies. We point to novel approaches for increasing confidence that semantic matching and empirical modeling used in these studies will yield accurate and valid measurements that can be combined in IDA. We also review findings about effect heterogeneity, emphasizing the importance of using etiologic and action theories to identify and evaluate sources of such effects. We note that all of the papers in this issue deserve careful attention, as they illustrate how prevention scientists are approaching the complexities of IDA and exploring novel methods for overcoming its challenges.

Keywords

References

  1. Multivariate Behav Res. 2014 Jun;49(3):214-231 [PMID: 25960575]
  2. Psychol Methods. 2017 Sep;22(3):507-526 [PMID: 27266798]
  3. Psychol Rev. 2003 Apr;110(2):203-219 [PMID: 12747522]
  4. Bull World Health Organ. 1991;69(5):541-8 [PMID: 1835674]
  5. Prev Sci. 2023 Nov;24(8):1595-1607 [PMID: 36441362]
  6. Prev Sci. 2023 Nov;24(8):1447-1458 [PMID: 35870094]
  7. Prev Sci. 2023 Nov;24(8):1622-1635 [PMID: 36057023]
  8. Prev Sci. 2023 Nov;24(8):1523-1534 [PMID: 36586068]
  9. Prev Sci. 2023 Nov;24(8):1510-1522 [PMID: 36478336]
  10. J Consult Clin Psychol. 1993 Dec;61(6):911-9 [PMID: 8113492]
  11. Prev Sci. 2023 Nov;24(8):1459-1482 [PMID: 37133684]
  12. Prev Sci. 2023 Nov;24(8):1648-1658 [PMID: 37726579]
  13. Prev Sci. 2023 Nov;24(8):1608-1621 [PMID: 35976524]
  14. Psychol Methods. 2022 Jul 25;: [PMID: 35878074]
  15. Lancet Psychiatry. 2019 Jun;6(6):518-527 [PMID: 31072801]
  16. Prev Sci. 2013 Apr;14(2):144-56 [PMID: 21360061]
  17. Prev Sci. 2018 Feb;19(Suppl 1):95-108 [PMID: 28243827]
  18. Prev Sci. 2018 Feb;19(Suppl 1):60-73 [PMID: 28434055]
  19. Psychol Methods. 2009 Jun;14(2):101-25 [PMID: 19485624]
  20. Prev Sci. 2023 Nov;24(8):1569-1580 [PMID: 35798992]
  21. Clin Trials. 2006;3(3):259-71 [PMID: 16895043]
  22. Prev Sci. 2018 Feb;19(Suppl 1):1-5 [PMID: 29368296]
  23. Prev Sci. 2023 Nov;24(8):1636-1647 [PMID: 37615885]
  24. Prev Sci. 2023 Nov;24(8):1483-1498 [PMID: 35994192]
  25. Prev Sci. 2023 Nov;24(8):1581-1594 [PMID: 36753042]
  26. Prev Sci. 2023 Feb;24(2):199-203 [PMID: 36378392]
  27. Stat Med. 2016 Jul 10;35(15):2485-502 [PMID: 26841367]
  28. Prev Sci. 2023 Nov;24(8):1435-1446 [PMID: 35796879]
  29. Psychol Methods. 2009 Jun;14(2):77-80 [PMID: 19485622]
  30. Prev Sci. 2023 Nov;24(8):1499-1509 [PMID: 36227399]
  31. Dev Psychopathol. 2016 Aug;28(3):689-706 [PMID: 27427800]
  32. Prev Sci. 2023 Nov;24(8):1535-1546 [PMID: 35994193]
  33. J Clin Child Adolesc Psychol. 2023 Jan-Feb;52(1):19-54 [PMID: 36040955]
  34. Addict Behav. 2019 Jul;94:65-73 [PMID: 30385076]
  35. Subst Abuse. 2023 Mar 11;17:11782218231157558 [PMID: 36923069]
  36. Prev Sci. 2023 Nov;24(8):1547-1557 [PMID: 36930405]
  37. Child Dev. 1994 Oct;65(5):1478-90 [PMID: 7982363]
  38. Psychol Methods. 2009 Jun;14(2):81-100 [PMID: 19485623]
  39. Prev Sci. 2018 Feb;19(Suppl 1):74-94 [PMID: 28013420]
  40. Prev Sci. 2022 Nov;23(8):1321-1332 [PMID: 36083435]
  41. Lancet. 1993 Feb 13;341(8842):418-22 [PMID: 8094183]
  42. J Child Psychol Psychiatry. 2013 Jul;54(7):810-2 [PMID: 23789914]
  43. Prev Sci. 2023 Nov;24(8):1659-1671 [PMID: 37060480]
  44. Prev Sci. 2023 Nov;24(8):1558-1568 [PMID: 35476247]
  45. Psychol Assess. 2019 Sep;31(9):1154-1167 [PMID: 31259571]
  46. Int Rev Psychiatry. 2007 Dec;19(6):617-31 [PMID: 18092240]
  47. J Consult Clin Psychol. 2021 Mar;89(3):166-175 [PMID: 33829805]

Grants

  1. P30 DA027828/NIDA NIH HHS
  2. R01 MH117598/NIMH NIH HHS
  3. R01 MH124718/NIMH NIH HHS
  4. 1R01MH117598/NIMH NIH HHS

MeSH Term

Humans
Psychometrics
Retrospective Studies
Causality
Research Design
Data Analysis