Statistical reporting inconsistencies in experimental philosophy.

Matteo Colombo, Georgi Duev, Michèle B Nuijten, Jan Sprenger
Author Information
  1. Matteo Colombo: Department of Philosophy and Tilburg Center for Logic, Ethics and Philosophy of Science (TiLPS), Tilburg University, Tilburg, The Netherlands.
  2. Georgi Duev: Center for Economic Research (CentER), Tilburg School of Economics and Management, Tiilburg University, Tilburg, The Netherlands.
  3. Michèle B Nuijten: Department of Methodology and Statistics, Tilburg University, Tilburg, The Netherlands.
  4. Jan Sprenger: Department of Philosophy and Educational Sciences and Center for Logic, Language and Cognition (LLC), Università degli Studi di Torino, Turin, Italy.

Abstract

Experimental philosophy (x-phi) is a young field of research in the intersection of philosophy and psychology. It aims to make progress on philosophical questions by using experimental methods traditionally associated with the psychological and behavioral sciences, such as null hypothesis significance testing (NHST). Motivated by recent discussions about a methodological crisis in the behavioral sciences, questions have been raised about the methodological standards of x-phi. Here, we focus on one aspect of this question, namely the rate of inconsistencies in statistical reporting. Previous research has examined the extent to which published articles in psychology and other behavioral sciences present statistical inconsistencies in reporting the results of NHST. In this study, we used the R package statcheck to detect statistical inconsistencies in x-phi, and compared rates of inconsistencies in psychology and philosophy. We found that rates of inconsistencies in x-phi are lower than in the psychological and behavioral sciences. From the point of view of statistical reporting consistency, x-phi seems to do no worse, and perhaps even better, than psychological science.

References

  1. Behav Res Methods. 2016 Dec;48(4):1205-1226 [PMID: 26497820]
  2. PLoS One. 2011;6(11):e26828 [PMID: 22073203]
  3. Perspect Psychol Sci. 2012 Nov;7(6):543-54 [PMID: 26168111]
  4. Behav Res Methods. 2011 Sep;43(3):666-78 [PMID: 21494917]
  5. Cognition. 2010 May;115(2):320-9 [PMID: 20116052]
  6. PLoS Med. 2005 Aug;2(8):e124 [PMID: 16060722]
  7. Perspect Psychol Sci. 2014 Nov;9(6):666-81 [PMID: 26186117]
  8. Psychol Sci. 2012 May 1;23(5):524-32 [PMID: 22508865]
  9. PLoS One. 2014 Jul 29;9(7):e103360 [PMID: 25072606]
  10. J Exp Psychol Gen. 2014 Apr;143(2):534-47 [PMID: 23855496]
  11. Q J Exp Psychol (Hove). 2013;66(12):2303-9 [PMID: 24205936]
  12. Cognition. 2004 Jul;92(3):B1-B12 [PMID: 15019555]
  13. Cognition. 2017 Apr;161:80-93 [PMID: 28157584]
  14. Psicothema. 2013;25(3):408-14 [PMID: 23910759]
  15. PLoS One. 2014 Dec 10;9(12 ):e114876 [PMID: 25493918]
  16. Psychol Sci. 2006 May;17(5):421-7 [PMID: 16683930]
  17. Science. 2015 Aug 28;349(6251):aac4716 [PMID: 26315443]

MeSH Term

Behavioral Sciences
Humans
Models, Statistical
Motivation
Philosophy
Psychology
Reference Standards
Reproducibility of Results
Research Design
Social Sciences
Software
Statistics as Topic

Word Cloud

Created with Highcharts 10.0.0inconsistenciesx-phiphilosophybehavioralsciencesstatisticalreportingpsychologypsychologicalresearchquestionsexperimentalNHSTmethodologicalratesExperimentalyoungfieldintersectionaimsmakeprogressphilosophicalusingmethodstraditionallyassociatednullhypothesissignificancetestingMotivatedrecentdiscussionscrisisraisedstandardsfocusoneaspectquestionnamelyratePreviousexaminedextentpublishedarticlespresentresultsstudyusedRpackagestatcheckdetectcomparedfoundlowerpointviewconsistencyseemsworseperhapsevenbetterscienceStatistical

Similar Articles

Cited By