Synchronization by the hand: the sight of gestures modulates low-frequency activity in brain responses to continuous speech.

Emmanuel Biau, Salvador Soto-Faraco
Author Information
  1. Emmanuel Biau: Multisensory Research Group, Center for Brain and Cognition, Universitat Pompeu Fabra Barcelona, Spain.
  2. Salvador Soto-Faraco: Multisensory Research Group, Center for Brain and Cognition, Universitat Pompeu Fabra Barcelona, Spain ; Institució Catalana de Recerca i Estudis Avançats (ICREA) Barcelona, Spain.

Abstract

During social interactions, speakers often produce spontaneous gestures to accompany their speech. These coordinated body movements convey communicative intentions, and modulate how listeners perceive the message in a subtle, but important way. In the present perspective, we put the focus on the role that congruent non-verbal information from beat gestures may play in the neural responses to speech. Whilst delta-theta oscillatory brain responses reflect the time-frequency structure of the speech signal, we argue that beat gestures promote phase resetting at relevant word onsets. This mechanism may facilitate the anticipation of associated acoustic cues relevant for prosodic/syllabic-based segmentation in speech perception. We report recently published data supporting this hypothesis, and discuss the potential of beats (and gestures in general) for further studies investigating continuous AV speech processing through low-frequency oscillations.

Keywords

References

  1. J Neurosci. 2008 Apr 9;28(15):3958-65 [PMID: 18400895]
  2. Hum Brain Mapp. 2009 Mar;30(3):1028-37 [PMID: 18412134]
  3. Neuron. 2007 Jun 21;54(6):1001-10 [PMID: 17582338]
  4. Brain Lang. 2013 Feb;124(2):143-52 [PMID: 23333667]
  5. Cortex. 2015 Jul;68:76-85 [PMID: 25595613]
  6. J Cogn Neurosci. 2007 Dec;19(12):1964-73 [PMID: 17892381]
  7. Trends Cogn Sci. 1999 Nov;3(11):419-429 [PMID: 10529797]
  8. Curr Biol. 2015 Jun 15;25(12 ):1649-53 [PMID: 26028433]
  9. Proc Natl Acad Sci U S A. 2005 Jan 25;102(4):1181-6 [PMID: 15647358]
  10. Science. 2008 Apr 4;320(5872):110-3 [PMID: 18388295]
  11. Nature. 1976 Dec 23-30;264(5588):746-8 [PMID: 1012311]
  12. Front Psychol. 2012 Mar 19;3:74 [PMID: 22457657]
  13. J Cogn Neurosci. 2010 Jul;22(7):1583-96 [PMID: 19583474]
  14. Neuropsychologia. 2013 Nov;51(13):2847-55 [PMID: 24060845]
  15. Proc Natl Acad Sci U S A. 2001 Nov 6;98(23):13367-72 [PMID: 11698688]
  16. J Neurosci. 2009 Dec 9;29(49):15564-74 [PMID: 20007480]
  17. Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130297 [PMID: 25092665]
  18. J Neurosci. 2015 Feb 4;35(5):1954-64 [PMID: 25653354]
  19. PLoS One. 2011;6(10):e25198 [PMID: 21998642]
  20. Nat Neurosci. 2012 Mar 18;15(4):511-7 [PMID: 22426255]
  21. Front Psychol. 2014 Jul 07;5:700 [PMID: 25071666]
  22. Front Psychol. 2012 Sep 06;3:320 [PMID: 22973251]
  23. J Cogn Neurosci. 2010 Apr;22(4):683-94 [PMID: 19413483]
  24. Trends Neurosci. 2009 Jan;32(1):9-18 [PMID: 19012975]
  25. PLoS Biol. 2013 Dec;11(12):e1001752 [PMID: 24391472]
  26. Lang Cogn Process. 2014;29(7):771-780 [PMID: 25018577]
  27. Trends Cogn Sci. 2012 Jul;16(7):390-8 [PMID: 22682813]
  28. Phonetica. 2009;66(1-2):113-26 [PMID: 19390234]
  29. Trends Cogn Sci. 2008 Mar;12(3):106-13 [PMID: 18280772]
  30. Psychol Sci. 2004 Feb;15(2):133-7 [PMID: 14738521]
  31. Neuropsychologia. 2015 Jun;72:27-42 [PMID: 25900470]

Grants

  1. 263145/European Research Council

Word Cloud

Created with Highcharts 10.0.0speechgesturesresponseslow-frequencybeatmaybrainrelevantbeatscontinuousoscillationssocialinteractionsspeakersoftenproducespontaneousaccompanycoordinatedbodymovementsconveycommunicativeintentionsmodulatelistenersperceivemessagesubtleimportantwaypresentperspectiveputfocusrolecongruentnon-verbalinformationplayneuralWhilstdelta-thetaoscillatoryreflecttime-frequencystructuresignalarguepromotephaseresettingwordonsetsmechanismfacilitateanticipationassociatedacousticcuesprosodic/syllabic-basedsegmentationperceptionreportrecentlypublisheddatasupportinghypothesisdiscusspotentialgeneralstudiesinvestigatingAVprocessingSynchronizationhand:sightmodulatesactivityEEGaudiovisual

Similar Articles

Cited By