Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

A neural generative model for joint learning topics and topic-specific word embeddings

Tools
- Tools
+ Tools

Zhu, Lixing, He, Yulan and Zhou, Deyu (2020) A neural generative model for joint learning topics and topic-specific word embeddings. Transactions of the Association for Computational Linguistics, 8 . pp. 471-485. doi:10.1162/tacl_a_00326

[img]
Preview
PDF
WRAP-neural-generative-model-joint-learning-topics-specific-embeddings-Zhu-2020.pdf - Published Version - Requires a PDF viewer.
Available under License Creative Commons Attribution 4.0.

Download (584Kb) | Preview
[img] PDF
WRAP-neural-generative-model-joint-learning-topics-topic-specific-word-embeddings-Zhu-2020.pdf - Accepted Version
Embargoed item. Restricted access to Repository staff only - Requires a PDF viewer.

Download (898Kb)
Official URL: https://doi.org/10.1162/tacl_a_00326

Request Changes to record.

Abstract

We propose a novel generative model to explore both local and global context for joint learning topics and topic-specific word embeddings. In particular, we assume that global latent topics are shared across documents; a word is generated by a hidden semantic vector encoding its contextual semantic meaning; and its context words are generated conditional on both the hidden semantic vector and global latent topics. Topics are trained jointly with the word embeddings. The trained model maps words to topic-dependent embeddings, which naturally addresses the issue of word polysemy. Experimental results show that the proposed model outperforms the word-level embedding methods in both word similarity evaluation and word sense disambiguation. Furthermore, the model also extracts more coherent topics compared to existing neural topic models or other models for joint learning of topics and word embeddings. Finally, the model can be easily integrated with existing deep contextualized word embedding learning methods to further improve the performance of downstream tasks such as sentiment classification.

Item Type: Journal Article
Subjects: Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
T Technology > TK Electrical engineering. Electronics Nuclear engineering
Divisions: Faculty of Science > Computer Science
Library of Congress Subject Headings (LCSH): Natural language processing (Computer science) , Neural networks (Computer science) , Embedded computer systems
Journal or Publication Title: Transactions of the Association for Computational Linguistics
Publisher: Association for Computational Linguistics
ISSN: 2307-387X
Official Date: 2020
Dates:
DateEvent
2020Available
20 April 2020Accepted
25 May 2020Modified
Date of first compliant deposit: 31 July 2020
Volume: 8
Page Range: pp. 471-485
DOI: 10.1162/tacl_a_00326
Status: Peer Reviewed
Publication Status: Published
Access rights to Published version: Open Access
Copyright Holders: © 2020 Association for Computational Linguistics
RIOXX Funder/Project Grant:
Project/Grant IDRIOXX Funder NameFunder ID
EP/T017112/1[EPSRC] Engineering and Physical Sciences Research Councilhttp://dx.doi.org/10.13039/501100000266
Chancellor’s International Scholarship University of Warwickhttp://dx.doi.org/10.13039/501100000741
2017YFB1002801Ministry of Science and Technology of the People's Republic of Chinahttp://dx.doi.org/10.13039/501100002855
61772132[NSFC] National Natural Science Foundation of Chinahttp://dx.doi.org/10.13039/501100001809
Related URLs:
  • Publisher
  • Publisher

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item
twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us