Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

Learning affective correspondence between music and image

Tools
- Tools
+ Tools

Verma, Gaurav , Dhekane , Eeshan G. and Guha, Tanaya (2019) Learning affective correspondence between music and image. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP, Brighton, 12-17 May 2019. Published in: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) doi:10.1109/ICASSP.2019.8683133 ISSN 2379-190X.

[img]
Preview
PDF
WRAP-learning-affective-correspondence-music-image-Guha-2019.pdf - Accepted Version - Requires a PDF viewer.

Download (2798Kb) | Preview
Official URL: https://doi.org/10.1109/ICASSP.2019.8683133

Request Changes to record.

Abstract

We introduce the problem of learning affective correspondence between audio (music) and visual data (images). For this task, a music clip and an image are considered similar (having true correspondence) if they have similar emotion content. In order to estimate this cross modal, emotion-centric similarity, we propose a deep neural network architecture that learns to project the data from the two modalities to a common representation space, and performs a binary classification task of predicting the affective correspondence (true or false). To facilitate the current study, we construct a large scale database containing more than 3,500 music clips and 85,000 images with three emotion classes (positive, neutral, negative). The proposed approach achieves 61.67%accuracy for the affective correspondence prediction task on this database, outperforming two relevant and competitive baselines. We also demonstrate that our net-work learns modality-specific representations of emotion (without explicitly being trained with emotion labels), which are useful foremotion recognition in individual modalities.

Item Type: Conference Item (Paper)
Subjects: B Philosophy. Psychology. Religion > BF Psychology
Divisions: Faculty of Science, Engineering and Medicine > Science > Computer Science
Library of Congress Subject Headings (LCSH): Music -- Psychological aspects, Imagery (Psychology)
Journal or Publication Title: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher: IEEE
ISSN: 2379-190X
Official Date: 17 April 2019
Dates:
DateEvent
17 April 2019Published
1 February 2019Accepted
DOI: 10.1109/ICASSP.2019.8683133
Status: Peer Reviewed
Publication Status: Published
Reuse Statement (publisher, data, author rights): © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Access rights to Published version: Restricted or Subscription Access
Date of first compliant deposit: 19 February 2019
Date of first compliant Open Access: 19 February 2019
Conference Paper Type: Paper
Title of Event: International Conference on Acoustics, Speech and Signal Processing (ICASSP
Type of Event: Conference
Location of Event: Brighton
Date(s) of Event: 12-17 May 2019
Related URLs:
  • Organisation
  • Organisation
  • Publisher

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics

twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us