Gui, Lin, Leng, Jia, Pergola, Gabriele, Zhou, Yu, Xu, Ruifeng and He, Yulan (2019) Neural topic model with reinforcement learning. In: 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, 3-7 Nov 2019. Published in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) pp. 3478-3483. doi:10.18653/v1/D19-1350

Preview	PDF WRAP-neural-topic-model-reinforcement-learning-Gui-2019.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (356Kb) | Preview
	PDF WRAP-neural-topic-model-reinforcement-learning-Gui-2019.pdf - Accepted Version Embargoed item. Restricted access to Repository staff only - Requires a PDF viewer. Download (578Kb)

Request Changes to record.

Abstract

In recent years, advances in neural variational inference have achieved many successes in text processing. Examples include neural topic models which are typically built upon variational autoencoder (VAE) with an objective of minimising the error of reconstructing original documents based on the learned latent topic vectors. However, minimising reconstruction errors does not necessarily lead to high quality topics. In this paper, we borrow the idea of reinforcement learning and incorporate topic coherence measures as reward signals to guide the learning of a VAE-based topic model. Furthermore, our proposed model is able to automatically separating background words dynamically from topic words, thus eliminating the pre-processing step of filtering infrequent and/or top frequent words, typically required for learning traditional topic models. Experimental results on the 20 Newsgroups and the NIPS datasets show superior performance both on perplexity and topic coherence measure compared to state-of-the-art neural topic models.

Item Type:	Conference Item (Paper)
Subjects:	Q Science > Q Science (General) Q Science > QA Mathematics
Divisions:	Faculty of Science > Computer Science
Library of Congress Subject Headings (LCSH):	Text processing (Computer science), Reinforcement learning -- Mathematical models, Neural networks (Computer science)
Journal or Publication Title:	Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Publisher:	Association for Computational Linguistics
Official Date:	November 2019
Dates:	Date Event November 2019 Published 12 February 2019 Accepted
Page Range:	pp. 3478-3483
DOI:	10.18653/v1/D19-1350
Status:	Peer Reviewed
Publication Status:	Published
Publisher Statement:	ACL materials are Copyright © 1963–2021 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.
Access rights to Published version:	Open Access
RIOXX Funder/Project Grant:	Project/Grant ID RIOXX Funder Name Funder ID 794196 Horizon 2020 Framework Programme http://dx.doi.org/10.13039/100010661 103652 Innovate UK http://dx.doi.org/10.13039/501100006041 U1636103, 61876053. [NSFC] National Natural Science Foundation of China http://dx.doi.org/10.13039/501100001809
Conference Paper Type:	Paper
Title of Event:	2019 Conference on Empirical Methods in Natural Language Processing
Type of Event:	Conference
Location of Event:	Hong Kong
Date(s) of Event:	3-7 Nov 2019
Related URLs:	Organisation Publisher

Request changes or add full text files to a record

Repository staff actions (login required)

View Item