Toward Cumulative Cognitive Science: A Comparison of Meta-Analysis, Mega-Analysis, and Hybrid Approaches.

Ezequiel Koile, Alejandrina Cristia
Author Information
  1. Ezequiel Koile: National Research University Higher School of Economics. ORCID
  2. Alejandrina Cristia: Laboratoire de Sciences Cognitives et Psycholinguistique, De´partement d'e´tudes cognitives, ENS, EHESS, CNRS, PSL University. ORCID

Abstract

There is increasing interest in cumulative approaches to science, in which instead of analyzing the results of individual papers separately, we integrate information qualitatively or quantitatively. One such approach is meta-analysis, which has over 50 years of literature supporting its usefulness, and is becoming more common in cognitive science. However, changes in technical possibilities by the widespread use of Python and R make it easier to fit more complex models, and even simulate missing data. Here we recommend the use of mega-analyses (based on the aggregation of data sets collected by independent researchers) and hybrid meta- mega-analytic approaches, for cases where raw data are available for some studies. We illustrate the three approaches using a rich test-retest data set of infants' speech processing as well as synthetic data. We discuss advantages and disadvantages of the three approaches from the viewpoint of a cognitive scientist contemplating their use, and limitations of this article, to be addressed in future work.

Keywords

References

  1. Ann Transl Med. 2016 Jan;4(2):30 [PMID: 26889483]
  2. Res Synth Methods. 2019 Sep;10(3):360-375 [PMID: 30523676]
  3. J Clin Epidemiol. 2021 Aug;136:227-234 [PMID: 34044099]
  4. Front Neuroinform. 2009 Sep 30;3:33 [PMID: 19826498]
  5. Infancy. 2007 Sep;12(2):119-145 [PMID: 33412746]
  6. Child Dev. 2014 Jul-Aug;85(4):1330-45 [PMID: 24320112]
  7. Perspect Psychol Sci. 2014 Nov;9(6):626-40 [PMID: 26186113]
  8. Cochrane Database Syst Rev. 2019 Oct 3;10:ED000142 [PMID: 31643080]
  9. J Appl Meas. 2003;4(4):298-308 [PMID: 14523251]
  10. Child Dev. 2018 Nov;89(6):1996-2009 [PMID: 29736962]
  11. Curr Dir Psychol Sci. 2020 Apr;29(2):199-206 [PMID: 32655212]
  12. Infancy. 2020 Jul;25(4):393-419 [PMID: 32744759]
  13. Stat Med. 2018 Dec 20;37(29):4404-4420 [PMID: 30101507]
  14. Front Neuroinform. 2019 Jan 08;12:102 [PMID: 30670959]
  15. Psychol Methods. 2019 Oct;24(5):571-575 [PMID: 31580141]
  16. Stat Med. 2017 Feb 28;36(5):855-875 [PMID: 27747915]
  17. Genet Epidemiol. 2014 May;38(4):369-78 [PMID: 24719363]
  18. Stat Med. 2020 Jul 10;39(15):2115-2137 [PMID: 32350891]
  19. Res Synth Methods. 2010 Jan;1(1):39-65 [PMID: 26056092]

Word Cloud

Created with Highcharts 10.0.0dataapproachesscienceusecumulativecognitivemega-analysesthreeeffectsincreasinginterestinsteadanalyzingresultsindividualpapersseparatelyintegrateinformationqualitativelyquantitativelyOneapproachmeta-analysis50yearsliteraturesupportingusefulnessbecomingcommonHoweverchangestechnicalpossibilitieswidespreadPythonRmakeeasierfitcomplexmodelsevensimulatemissingrecommendbasedaggregationsetscollectedindependentresearchershybridmeta-mega-analyticcasesrawavailablestudiesillustrateusingrichtest-retestsetinfants'speechprocessingwellsyntheticdiscussadvantagesdisadvantagesviewpointscientistcontemplatinglimitationsarticleaddressedfutureworkTowardCumulativeCognitiveScience:ComparisonMeta-AnalysisMega-AnalysisHybridApproachessimulationfixedmeta-analysesopenrandom

Similar Articles

Cited By