The Library
Learning affective correspondence between music and image
Tools
Tools
Tools
Verma, Gaurav , Dhekane , Eeshan G. and Guha, Tanaya (2019) Learning affective correspondence between music and image. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP, Brighton, 12-17 May 2019
|
PDF
WRAP-learning-affective-correspondence-music-image-Guha-2019.pdf - Accepted Version - Requires a PDF viewer. Download (2798Kb) | Preview |
Abstract
We introduce the problem of learning affective correspondence between audio (music) and visual data (images). For this task, a music clip and an image are considered similar (having true correspondence) if they have similar emotion content. In order to estimate this cross modal, emotion-centric similarity, we propose a deep neural network architecture that learns to project the data from the two modalities to a common representation space, and performs a binary classification task of predicting the affective correspondence (true or false). To facilitate the current study, we construct a large scale database containing more than 3,500 music clips and 85,000 images with three emotion classes (positive, neutral, negative). The proposed approach achieves 61.67%accuracy for the affective correspondence prediction task on this database, outperforming two relevant and competitive baselines. We also demonstrate that our net-work learns modality-specific representations of emotion (without explicitly being trained with emotion labels), which are useful foremotion recognition in individual modalities.
| Item Type: | Conference Item (Paper) | ||||
|---|---|---|---|---|---|
| Subjects: | B Philosophy. Psychology. Religion > BF Psychology | ||||
| Divisions: | Faculty of Science > Computer Science | ||||
| Library of Congress Subject Headings (LCSH): | Music -- Psychological aspects, Imagery (Psychology) | ||||
| Publisher: | IEEE | ||||
| Official Date: | 1 February 2019 | ||||
| Dates: |
|
||||
| Date of first compliant deposit: | 19 February 2019 | ||||
| Status: | Peer Reviewed | ||||
| Publication Status: | Forthcoming | ||||
| Conference Paper Type: | Paper | ||||
| Title of Event: | International Conference on Acoustics, Speech and Signal Processing (ICASSP | ||||
| Type of Event: | Conference | ||||
| Location of Event: | Brighton | ||||
| Date(s) of Event: | 12-17 May 2019 | ||||
| Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
