The Library
An evaluation of multi-probe locality sensitive hashing for computing similarities over web-scale query logs
Tools
Cormode, Graham, Dasgupta, Anirban, Goyal, Amit and Lee, Chi Hoon (2018) An evaluation of multi-probe locality sensitive hashing for computing similarities over web-scale query logs. PLoS One, 13 (1). e0191175. doi:10.1371/journal.pone.0191175 ISSN 1932-6203.
|
PDF
WRAP-evaluation-multi-probe-locality-sensitive-hashing-web-scale-logs-Cormode-2018.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (2523Kb) | Preview |
Official URL: http://doi.org/10.1371/journal.pone.0191175
Abstract
Many modern applications of AI such as web search, mobile browsing, image processing, and natural language processing rely on finding similar items from a large database of complex objects. Due to the very large scale of data involved (e.g., users’ queries from commercial search engines), computing such near or nearest neighbors is a non-trivial task, as the computational cost grows significantly with the number of items. To address this challenge, we adopt Locality Sensitive Hashing (a.k.a, LSH) methods and evaluate four variants in a distributed computing environment (specifically, Hadoop). We identify several optimizations which improve performance, suitable for deployment in very large scale settings. The experimental results demonstrate our variants of LSH achieve the robust performance with better recall compared with “vanilla” LSH, even when using the same amount of space.
Item Type: | Journal Article | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software | ||||||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||||||||
SWORD Depositor: | Library Publications Router | ||||||||||||
Library of Congress Subject Headings (LCSH): | Hashing (Computer science), Keyword searching, Electronic information resource searching, Search engines | ||||||||||||
Journal or Publication Title: | PLoS One | ||||||||||||
Publisher: | Public Library of Science | ||||||||||||
ISSN: | 1932-6203 | ||||||||||||
Official Date: | 18 January 2018 | ||||||||||||
Dates: |
|
||||||||||||
Volume: | 13 | ||||||||||||
Number: | 1 | ||||||||||||
Article Number: | e0191175 | ||||||||||||
DOI: | 10.1371/journal.pone.0191175 | ||||||||||||
Status: | Peer Reviewed | ||||||||||||
Publication Status: | Published | ||||||||||||
Reuse Statement (publisher, data, author rights): | ** From PLOS via Jisc Publications Router. ** History: received 05-05-2017; accepted 03-12-2017; collection 2018; epub 18-01-2018. ** Licence for this article: http://creativecommons.org/licenses/by/4.0/ | ||||||||||||
Access rights to Published version: | Open Access (Creative Commons) | ||||||||||||
Date of first compliant deposit: | 19 January 2018 | ||||||||||||
Date of first compliant Open Access: | 19 January 2018 | ||||||||||||
RIOXX Funder/Project Grant: |
|
||||||||||||
Contributors: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year