The Library
A multimodal mixture-of-experts model for dynamic emotion prediction in movies
Tools
Goyal, Ankit, Kumar, Naveen, Guha, Tanaya and Narayanan, Shrikanth S. (2016) A multimodal mixture-of-experts model for dynamic emotion prediction in movies. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, 20-25 March 2016 pp. 2822-2826. ISBN 9781479999880. doi:10.1109/ICASSP.2016.7472192 ISSN 2379-190X.
|
PDF
WRAP-multimodal-mixture-of-experts-model-dynamic-emotion-prediction-movies-Guha-2016.pdf - Accepted Version - Requires a PDF viewer. Download (751Kb) | Preview |
Official URL: http://dx.doi.org/10.1109/ICASSP.2016.7472192
Abstract
This paper addresses the problem of continuous emotion prediction in movies from multimodal cues. The rich emotion content in movies is inherently multimodal, where emotion is evoked through both audio (music, speech) and video modalities. To capture such affective information, we put forth a set of audio and video features that includes several novel features such as, Video Compressibility and Histogram of Facial Area (HFA). We propose a Mixture of Experts (MoE)-based fusion model that dynamically combines information from the audio and video modalities for predicting the emotion evoked in movies. A learning module, based on hard Expectation-Maximization (EM) algorithm, is presented for the MoE model. Experiments on a database of popular movies demonstrate that our MoE-based fusion method outperforms popular fusion strategies (e.g. early and late fusion) in the context of dynamic emotion prediction.
Item Type: | Conference Item (Paper) | ||||||
---|---|---|---|---|---|---|---|
Subjects: | P Language and Literature > PN Literature (General) > PN1993 Motion Pictures Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software |
||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||
Library of Congress Subject Headings (LCSH): | Emotions in motion pictures -- Mathematical models | ||||||
Publisher: | IEEE | ||||||
ISBN: | 9781479999880 | ||||||
ISSN: | 2379-190X | ||||||
Book Title: | 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | ||||||
Official Date: | 2016 | ||||||
Dates: |
|
||||||
Page Range: | pp. 2822-2826 | ||||||
DOI: | 10.1109/ICASSP.2016.7472192 | ||||||
Status: | Peer Reviewed | ||||||
Publication Status: | Published | ||||||
Reuse Statement (publisher, data, author rights): | © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | ||||||
Access rights to Published version: | Restricted or Subscription Access | ||||||
Date of first compliant deposit: | 30 October 2018 | ||||||
Date of first compliant Open Access: | 31 October 2018 | ||||||
Conference Paper Type: | Paper | ||||||
Title of Event: | 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | ||||||
Type of Event: | Conference | ||||||
Location of Event: | Shanghai, China | ||||||
Date(s) of Event: | 20-25 March 2016 |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year