The Library
VISIR : visual and semantic image label refinement
Tools
Chowdhury, Sreyasi Nag, Tandon, Niket, Ferhatosmanoglu, Hakan and Weikum, Gerhard (2018) VISIR : visual and semantic image label refinement. In: 11th ACM International Conference on Web Search and Data Mining (WSDM) 2018, Marina Del Rey, CA, USA, 5–9 Feb 2018. Published in: Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM) 2018 pp. 117-125. ISBN 9781450355810 . doi:10.1145/3159652.3159693
|
PDF
WRAP-VISIR-visual-semantic-image-label-refinement-Ferhatosmanoglu-2017.pdf - Accepted Version - Requires a PDF viewer. Download (3084Kb) | Preview |
Official URL: http://dx.doi.org/10.1145/3159652.3159693
Abstract
The social media explosion has populated the Internet with a wealth of images. There are two existing paradigms for image retrieval: 1) content-based image retrieval (CBIR), which has traditionally used visual features for similarity search (e.g., SIFT features), and 2) tag-based image retrieval (TBIR), which has relied on user tagging (e.g., Flickr tags). CBIR now gains semantic expressiveness by advances in deep-learning-based detection of visual labels. TBIR benefits from query-and-click logs to automatically infer more informative labels. However, learning-based tagging still yields noisy labels and is restricted to concrete objects, missing out on generalizations and abstractions. Click-based tagging is limited to terms that appear in the textual context of an image or in queries that lead to a click. This paper addresses the above limitations by semantically refining and expanding the labels suggested by learning-based object detection. We consider the semantic coherence between the labels for different objects, leverage lexical and commonsense knowledge, and cast the label assignment into a constrained optimization problem solved by an integer linear program. Experiments show that our method, called VISIR, improves the quality of the state-of-the-art visual labeling tools like LSDA and YOLO.
Item Type: | Conference Item (Paper) | ||||||
---|---|---|---|---|---|---|---|
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||
Journal or Publication Title: | Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM) 2018 | ||||||
Publisher: | ACM | ||||||
ISBN: | 9781450355810 | ||||||
Official Date: | 2018 | ||||||
Dates: |
|
||||||
Page Range: | pp. 117-125 | ||||||
DOI: | 10.1145/3159652.3159693 | ||||||
Status: | Peer Reviewed | ||||||
Publication Status: | Published | ||||||
Access rights to Published version: | Restricted or Subscription Access | ||||||
Date of first compliant deposit: | 5 December 2017 | ||||||
Date of first compliant Open Access: | 21 December 2017 | ||||||
Conference Paper Type: | Paper | ||||||
Title of Event: | 11th ACM International Conference on Web Search and Data Mining (WSDM) 2018 | ||||||
Type of Event: | Conference | ||||||
Location of Event: | Marina Del Rey, CA, USA | ||||||
Date(s) of Event: | 5–9 Feb 2018 | ||||||
Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year