The Library
Learning affective correspondence between music and image
Tools
Tools
Tools
Verma, Gaurav , Dhekane , Eeshan G. and Guha, Tanaya (2019) Learning affective correspondence between music and image. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP, Brighton, 12-17 May 2019. Published in: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ISSN 2379-190X. doi:10.1109/ICASSP.2019.8683133
|
PDF
WRAP-learning-affective-correspondence-music-image-Guha-2019.pdf - Accepted Version - Requires a PDF viewer. Download (2798Kb) | Preview |
Official URL: https://doi.org/10.1109/ICASSP.2019.8683133
Abstract
We introduce the problem of learning affective correspondence between audio (music) and visual data (images). For this task, a music clip and an image are considered similar (having true correspondence) if they have similar emotion content. In order to estimate this cross modal, emotion-centric similarity, we propose a deep neural network architecture that learns to project the data from the two modalities to a common representation space, and performs a binary classification task of predicting the affective correspondence (true or false). To facilitate the current study, we construct a large scale database containing more than 3,500 music clips and 85,000 images with three emotion classes (positive, neutral, negative). The proposed approach achieves 61.67%accuracy for the affective correspondence prediction task on this database, outperforming two relevant and competitive baselines. We also demonstrate that our net-work learns modality-specific representations of emotion (without explicitly being trained with emotion labels), which are useful foremotion recognition in individual modalities.
| Item Type: | Conference Item (Paper) | ||||||
|---|---|---|---|---|---|---|---|
| Subjects: | B Philosophy. Psychology. Religion > BF Psychology | ||||||
| Divisions: | Faculty of Science > Computer Science | ||||||
| Library of Congress Subject Headings (LCSH): | Music -- Psychological aspects, Imagery (Psychology) | ||||||
| Journal or Publication Title: | ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | ||||||
| Publisher: | IEEE | ||||||
| ISSN: | 2379-190X | ||||||
| Official Date: | 17 April 2019 | ||||||
| Dates: |
|
||||||
| DOI: | 10.1109/ICASSP.2019.8683133 | ||||||
| Status: | Peer Reviewed | ||||||
| Publication Status: | Published | ||||||
| Publisher Statement: | © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | ||||||
| Access rights to Published version: | Restricted or Subscription Access | ||||||
| Conference Paper Type: | Paper | ||||||
| Title of Event: | International Conference on Acoustics, Speech and Signal Processing (ICASSP | ||||||
| Type of Event: | Conference | ||||||
| Location of Event: | Brighton | ||||||
| Date(s) of Event: | 12-17 May 2019 | ||||||
| Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
