Comparing the use of open and closed questions for Web-based measures of the continued-influence effect.

Saoirse Connor Desai, Stian Reimers
Author Information
  1. Saoirse Connor Desai: Department of Psychology, City, University of London, Northampton Square, London, EC1V 0HB, UK. saoirse.connor-desai@city.ac.uk. ORCID
  2. Stian Reimers: Department of Psychology, City, University of London, Northampton Square, London, EC1V 0HB, UK.

Abstract

Open-ended questions, in which participants write or type their responses, are used in many areas of the behavioral sciences. Although effective in the lab, they are relatively untested in online experiments, and the quality of responses is largely unexplored. Closed-ended questions are easier to use online because they generally require only single key- or mouse-press responses and are less cognitively demanding, but they can bias the responses. We compared the data quality obtained using open and closed response formats using the continued-influence effect (CIE), in which participants read a series of statements about an unfolding event, one of which is unambiguously corrected later. Participants typically continue to refer to the corrected misinformation when making inferential statements about the event. We implemented this basic procedure online (Exp. 1A, n = 78), comparing standard open-ended responses to an alternative procedure using closed-ended responses (Exp. 1B, n = 75). Finally, we replicated these findings in a larger preregistered study (Exps. 2A and 2B, n = 323). We observed the CIE in all conditions: Participants continued to refer to the misinformation following a correction, and their references to the target misinformation were broadly similar in number across open- and closed-ended questions. We found that participants' open-ended responses were relatively detailed (including an average of 75 characters for inference questions), and almost all responses attempted to address the question. The responses were faster, however, for closed-ended questions. Overall, we suggest that with caution it may be possible to use either method for gathering CIE data.

Keywords

References

  1. Psychol Sci Public Interest. 2012 Dec;13(3):106-31 [PMID: 26173286]
  2. Neuropsychologia. 2017 Nov;106:216-224 [PMID: 28987910]
  3. Psychon Bull Rev. 2018 Oct;25(5):1968-1972 [PMID: 28744765]
  4. Mem Cognit. 2010 Dec;38(8):1087-100 [PMID: 21156872]
  5. J Exp Psychol Gen. 2015 Oct;144(5):993-1002 [PMID: 26301795]
  6. Trends Cogn Sci. 2017 Oct;21(10):736-748 [PMID: 28803699]
  7. Psychon Bull Rev. 2011 Jun;18(3):570-8 [PMID: 21359617]
  8. Neuropsychologia. 2000;38(10):1333-41 [PMID: 10869576]
  9. Behav Res Methods. 2011 Sep;43(3):800-13 [PMID: 21437749]
  10. Psychon Bull Rev. 2012 Oct;19(5):847-57 [PMID: 22829343]
  11. Health Policy. 2004 Jun;68(3):289-98 [PMID: 15113640]
  12. Biometrics. 1977 Mar;33(1):159-74 [PMID: 843571]
  13. Behav Res Methods. 2015 Jun;47(2):309-27 [PMID: 24903687]
  14. Behav Res Methods. 2017 Oct;49(5):1615-1620 [PMID: 28176258]
  15. J Pers Soc Psychol. 2009 Jan;96(1):32-44 [PMID: 19210062]
  16. Psychol Sci. 2017 Nov;28(11):1531-1546 [PMID: 28895452]
  17. Behav Res Methods. 2014 Dec;46(4):1023-31 [PMID: 24356996]
  18. PLoS One. 2013;8(3):e57410 [PMID: 23516406]
  19. Behav Res Methods. 2014 Mar;46(1):95-111 [PMID: 23670340]
  20. Can J Exp Psychol. 2013 Sep;67(3):215-27 [PMID: 24041303]
  21. Med Decis Making. 2003 Mar-Apr;23(2):150-9 [PMID: 12693877]
  22. Q J Exp Psychol (Hove). 2011 Feb;64(2):283-310 [PMID: 20694936]
  23. Behav Res Methods. 2007 Aug;39(3):365-70 [PMID: 17958146]
  24. J Exp Psychol Learn Mem Cogn. 2016 Jan;42(1):62-74 [PMID: 26147670]

MeSH Term

Adult
Bias
Communication
Female
Humans
Internet
Male
Middle Aged
Writing
Young Adult

Word Cloud

Created with Highcharts 10.0.0responsesquestionsonlineuseusingeffectCIEmisinformationn=closed-endedOpen-endedparticipantsrelativelyqualityClosed-endeddataopenclosedformatscontinued-influencestatementseventcorrectedParticipantsreferprocedureExpopen-ended75Web-basedwritetypeusedmanyareasbehavioralsciencesAlthougheffectivelabuntestedexperimentslargelyunexploredeasiergenerallyrequiresinglekey-mouse-presslesscognitivelydemandingcanbiascomparedobtainedresponsereadseriesunfoldingoneunambiguouslylatertypicallycontinuemakinginferentialimplementedbasic1A78comparingstandardalternative1BFinallyreplicatedfindingslargerpreregisteredstudyExps2A2B323observedconditions:continuedfollowingcorrectionreferencestargetbroadlysimilarnumberacrossopen-foundparticipants'detailedincludingaveragecharactersinferencealmostattemptedaddressquestionfasterhoweverOverallsuggestcautionmaypossibleeithermethodgatheringComparingmeasuresContinuedinfluenceMisinformationResponse

Similar Articles

Cited By