The Library
Leveraging ChatGPT in pharmacovigilance event extraction : an empirical study
Tools
Sun, Zhaoyue, Pergola, Gabriele, Wallace, Byron C and He, Yulan (2024) Leveraging ChatGPT in pharmacovigilance event extraction : an empirical study. In: 18th Conference of the European Chapter of the Association for Computational Linguistics, Malta, 17-22 Mar 2024 (In Press)
|
PDF
WRAP-Leveraging-ChatGPT-pharmacovigilance-event-extraction-empirical-study-24.pdf - Accepted Version - Requires a PDF viewer. Download (670Kb) | Preview |
Abstract
With the advent of large language models (LLMs), there has been growing interest in exploring their potential for medical applications. This research aims to investigate the ability of LLMs, specifically ChatGPT, in the context of pharmacovigilance event extraction, of which the main goal is to identify and extract adverse events or potential therapeutic events from textual medical sources. We conduct extensive experiments to assess the performance of ChatGPT in the pharmacovigilance event extraction task, employing various prompts and demonstration selection strategies. The findings demonstrate that while ChatGPT demonstrates reasonable performance with appropriate demonstration selection strategies, it still falls short compared to fully fine-tuned small models. Additionally, we explore the potential of leveraging ChatGPT for data augmentation. However, our investigation reveals that the inclusion of synthesized data into fine-tuning may lead to a decrease in performance, possibly attributed to noise in the ChatGPT-generated labels. To mitigate this, we explore different filtering strategies and find that, with the proper approach, more stable performance can be achieved, although constant improvement remains elusive.
Item Type: | Conference Item (Paper) | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software R Medicine > R Medicine (General) R Medicine > RM Therapeutics. Pharmacology |
||||||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||||||||
Library of Congress Subject Headings (LCSH): | Natural language processing (Computer science), Artificial intelligence, Artificial intelligence -- Medical applications , Pharmacovigilance , Pharmacovigilance -- Data processing | ||||||||||||
Official Date: | March 2024 | ||||||||||||
Dates: |
|
||||||||||||
Status: | Peer Reviewed | ||||||||||||
Publication Status: | In Press | ||||||||||||
Date of first compliant deposit: | 8 March 2024 | ||||||||||||
Date of first compliant Open Access: | 11 March 2024 | ||||||||||||
RIOXX Funder/Project Grant: |
|
||||||||||||
Conference Paper Type: | Paper | ||||||||||||
Title of Event: | 18th Conference of the European Chapter of the Association for Computational Linguistics | ||||||||||||
Type of Event: | Conference | ||||||||||||
Location of Event: | Malta | ||||||||||||
Date(s) of Event: | 17-22 Mar 2024 | ||||||||||||
Related URLs: | |||||||||||||
Open Access Version: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year