The Library
Dynamic emotion modeling with learnable graphs and graph inception network
Tools
Shirian, Amir, Tripathi, Subarna and Guha, Tanaya (2021) Dynamic emotion modeling with learnable graphs and graph inception network. IEEE Transactions on Multimedia, 24 . pp. 780-790. doi:10.1109/TMM.2021.3059169 ISSN 1520-9210.
|
PDF
WRAP-dynamic-emotion-modeling-learnable-graphs-graph-inception-network-Guha-2021.pdf - Accepted Version - Requires a PDF viewer. Download (4039Kb) | Preview |
Official URL: http://dx.doi.org/10.1109/TMM.2021.3059169
Abstract
Human emotion is expressed, perceived and captured using a variety of dynamic data modalities, such as speech (verbal), videos (facial expressions) and motion sensors (body gestures). We propose a generalized approach to emotion recognition that can adapt across modalities by modeling dynamic data as structured graphs. The motivation behind the graph approach is to build compact models without compromising on performance. To alleviate the problem of optimal graph construction, we cast this as a joint graph learning and classification task. To this end, we present the learnable graph inception network (L-GrIN) that jointly learns to recognize emotion and to identify the underlying graph structure in the dynamic data. Our architecture comprises multiple novel components: a new graph convolution operation, a graph inception layer, learnable adjacency, and a learnable pooling function that yields a graph-level embedding. We evaluate the proposed architecture on five benchmark emotion recognition databases spanning three different modalities (video, audio, motion capture), where each database captures one of the following emotional cues: facial expressions, speech and body gestures. We achieve state-of-the-art performance on all five databases outperforming several competitive baselines and relevant existing methods. Our graph architecture shows superior performance with significantly fewer parameters (compared to convolutional or recurrent neural networks) promising its applicability to resource-constrained devices. Our code is available at /github.com/AmirSh15/graph_emotion_recognition.
Item Type: | Journal Article | ||||||
---|---|---|---|---|---|---|---|
Subjects: | Q Science > Q Science (General) Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software |
||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||
Library of Congress Subject Headings (LCSH): | Graph theory -- Data processing, Neural networks (Computer science), Machine learning, Emotion recognition -- Computer programs, Human face recognition (Computer science), Human-computer interaction | ||||||
Journal or Publication Title: | IEEE Transactions on Multimedia | ||||||
Publisher: | IEEE | ||||||
ISSN: | 1520-9210 | ||||||
Official Date: | 15 February 2021 | ||||||
Dates: |
|
||||||
Volume: | 24 | ||||||
Page Range: | pp. 780-790 | ||||||
DOI: | 10.1109/TMM.2021.3059169 | ||||||
Status: | Peer Reviewed | ||||||
Publication Status: | Published | ||||||
Reuse Statement (publisher, data, author rights): | © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | ||||||
Access rights to Published version: | Restricted or Subscription Access | ||||||
Date of first compliant deposit: | 17 February 2021 | ||||||
Date of first compliant Open Access: | 18 February 2021 |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year