Somandepalli, Krishna, Kumar, Naveen, Guha, Tanaya and Narayanan, Shrikanth S. (2018) Unsupervised discovery of character dictionaries in animation movies. IEEE Transactions on Multimedia, 20 (3). pp. 539-551. doi:10.1109/TMM.2017.2745712 ISSN 1520-9210.

Preview

PDF WRAP-unsupervised-discovery-character-dictionaries-animation-movies-Guha-2018.pdf - Accepted Version - Requires a PDF viewer. Download (19Mb) | Preview

Request Changes to record.

Abstract

Automatic content analysis of animation movies can enable an objective understanding of character (actor) representations and their portrayals. It can also help illuminate potential markers of unconscious biases and their impact. However, multimedia analysis of movie content has predominantly focused on live-action features. A dearth of multimedia research in this field is because of the complexity and heterogeneity in the design of animated characters-an extremely challenging problem to be generalized by a single method or model. In this paper, we address the problem of automatically discovering characters in animation movies as a first step toward automatic character labeling in these media. Movie-specific character dictionaries can act as a powerful first step for subsequent content analysis at scale. We propose an unsupervised approach which requires no prior information about the characters in a movie. We first use a deep neural network-based object detector that is trained on natural images to identify a set of initial character candidates. These candidates are further pruned using saliency constraints and visual object tracking. A character dictionary per movie is then generated from exemplars obtained by clustering these candidates. We are able to identify both anthropomorphic and nonanthropomorphic characters in a dataset of 46 animation movies with varying composition and character design. Our results indicate high precision and recall of the automatically detected characters compared to human-annotated ground truth, demonstrating the generalizability of our approach.

Item Type:	Journal Article
Subjects:	Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Divisions:	Faculty of Science, Engineering and Medicine > Science > Computer Science
Library of Congress Subject Headings (LCSH):	Animated films, Cluster analysis -- Data processing, Content analysis (Communication) -- Data processing, Characters and characteristics in motion pictures -- Data processing, Neural networks (Computer science), Automatic tracking
Journal or Publication Title:	IEEE Transactions on Multimedia
Publisher:	IEEE
ISSN:	1520-9210
Official Date:	March 2018
Dates:	Date Event March 2018 Published 29 August 2017 Available 9 July 2017 Accepted 28 November 2016 Submitted
Volume:	20
Number:	3
Page Range:	pp. 539-551
DOI:	10.1109/TMM.2017.2745712
Status:	Peer Reviewed
Publication Status:	Published
Reuse Statement (publisher, data, author rights):	© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Access rights to Published version:	Restricted or Subscription Access
Date of first compliant deposit:	11 October 2018
Date of first compliant Open Access:	12 October 2018
RIOXX Funder/Project Grant:	Project/Grant ID RIOXX Funder Name Funder ID UNSPECIFIED National Science Foundation http://dx.doi.org/10.13039/100000001

Request changes or add full text files to a record

Repository staff actions (login required)

View Item

Downloads