The Library
Multimodal language processing : how preceding discourse constrains gesture interpretation and affects gesture integration when gestures do not synchronise with semantic affiliates
Tools
Tools
Tools
Fritz, Isabella, Kita, Sotaro, Littlemore, Jeannette and Krott, Andrea (2021) Multimodal language processing : how preceding discourse constrains gesture interpretation and affects gesture integration when gestures do not synchronise with semantic affiliates. Journal of Memory and Language, 117 . 104191. doi:10.1016/j.jml.2020.104191 ISSN 0749-596X.
|
PDF
WRAP-multimodal-language-processing-preceding-discourse-constrains-gesture-interpretation-Kita-2020.pdf - Accepted Version - Requires a PDF viewer. Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0. Download (1784Kb) | Preview |
|
|
PDF
WRAP-supplementary-material-Kita-2020.pdf - Supplemental Material - Requires a PDF viewer. Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0. Download (179Kb) | Preview |
Official URL: https://doi.org/10.1016/j.jml.2020.104191
Abstract
Previous studies have suggested that a co-speech gesture needs to be synchronous with semantically related speech (semantic affiliates) for its successful semantic integration into a discourse model because co-speech gestures are often highly ambiguous on their own. But not all gestures synchronise with semantic affiliates, some precede them. The current study tested whether the interpretation of a gesture that does not synchronise with its semantic affiliate can be constrained by preceding verbal discourse and integrated into a recipient’s discourse model. A behavioural experiment (Experiment 1) showed that related discourse information can indeed constrain recipients’ interpretations of such gestures. Results from an ERP experiment (Experiment 2) confirmed that synchronisation between gesture and semantic affiliate is not essential in order for the gesture to become part of a discourse model, but only if the preceding context constrains the gesture’s meaning. In this case, we found evidence for post-semantic integration (P600, time-locked to the gesture’s semantic affiliate).
| Item Type: | Journal Article | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Subjects: | B Philosophy. Psychology. Religion > BF Psychology P Language and Literature > P Philology. Linguistics |
||||||||
| Divisions: | Faculty of Science, Engineering and Medicine > Science > Psychology | ||||||||
| Library of Congress Subject Headings (LCSH): | Speech and gesture -- Research, Communication -- Research, Semantics | ||||||||
| Journal or Publication Title: | Journal of Memory and Language | ||||||||
| Publisher: | Academic Press | ||||||||
| ISSN: | 0749-596X | ||||||||
| Official Date: | April 2021 | ||||||||
| Dates: |
|
||||||||
| Volume: | 117 | ||||||||
| Article Number: | 104191 | ||||||||
| DOI: | 10.1016/j.jml.2020.104191 | ||||||||
| Status: | Peer Reviewed | ||||||||
| Publication Status: | Published | ||||||||
| Access rights to Published version: | Restricted or Subscription Access | ||||||||
| Date of first compliant deposit: | 9 November 2020 | ||||||||
| Date of first compliant Open Access: | 24 December 2021 | ||||||||
| RIOXX Funder/Project Grant: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
