The Library
Replicability and reproducibility of predictive models for diagnosis of depression among young adults using Electronic Health Records
Tools
Nickson, David, Singmann, Henrik , Meyer, Caroline, Toro, Carla T. and Walasek, Lukasz (2023) Replicability and reproducibility of predictive models for diagnosis of depression among young adults using Electronic Health Records. BMC Diagnostic and Prognostic Research, 7 . 25. doi:10.21203/rs.3.rs-3104286/v1 ISSN 2397-7523.
|
PDF
s41512-023-00160-2.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (1727Kb) | Preview |
|
PDF
Replicability and reproducibility of predictive models for diagnosis of depression among young adults using Electronic Health Records.pdf - Submitted Version Embargoed item. Restricted access to Repository staff only - Requires a PDF viewer. Download (1111Kb) |
Official URL: https://doi.org/10.21203/rs.3.rs-3104286/v1
Abstract
Background
Recent advances in machine learning combined with the growing availability of digitized health records offer new opportunities for improving early diagnosis of depression. An emerging body of research shows that Electronic Health Records can be used to accurately predict cases of depression on the basis of individual’s primary care records. The successes of these studies are undeniable, but there is a growing concern that their results may not be replicable, which could cast doubt on their clinical usefulness.
Methods
To address this issue in the present paper, we set out to reproduce and replicate the work by Nichols et al. (2018), who trained predictive models of depression among young adults using Electronic Healthcare Records. Our contribution consists of three parts. First, we attempt to replicate the methodology used by the original authors, acquiring the same set of primary health records and reproducing their data processing and analysis. Second, we test models presented in the original paper on our own data, thus providing out of sample prediction of the predictive models. Third, we extend past work by considering several novel machine learning approaches in an attempt to improve the predictive accuracy achieved in the original work.
Results
In summary, our results demonstrate that the work of Nichols et al. is largely reproducible and replicable. This was the case both for the replication of the original model and the out of sample replication applying NRCBM coefficients to our new EHRs data. Although alternative predictive models did not improve model performance over standard logistic regression, our results indicate that stepwise variable selection is not stable even in the case of large data sets.
Conclusion
We discuss the challenges associated with the research on mental health and Electronic Health Records, including the need to produce interpretable and robust models. We demonstrated some potential issues associated with the reliance on EHRs, including changes in the regulations and guidelines (such as the QOF guidelines in the UK) and reliance on visits to GP as a predictor of specific disorders.
Item Type: | Journal Article | ||||||||
---|---|---|---|---|---|---|---|---|---|
Subjects: | B Philosophy. Psychology. Religion > BF Psychology Q Science > Q Science (General) R Medicine > R Medicine (General) R Medicine > RC Internal medicine R Medicine > RJ Pediatrics |
||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Psychology Faculty of Science, Engineering and Medicine > Medicine > Warwick Medical School Faculty of Science, Engineering and Medicine > Engineering > WMG (Formerly the Warwick Manufacturing Group) |
||||||||
Library of Congress Subject Headings (LCSH): | Depression, Mental, Depression in adolescence , Depression, Mental -- Diagnosis -- Simulation methods, Depression in adolescence -- Diagnosis -- Simulation methods, Machine learning, Medical records -- Data processing, Depressed persons , Child mental health services, Teenagers -- Mental health services, Cognitive psychology | ||||||||
Journal or Publication Title: | BMC Diagnostic and Prognostic Research | ||||||||
Publisher: | Springer Nature | ||||||||
ISSN: | 2397-7523 | ||||||||
Official Date: | 5 December 2023 | ||||||||
Dates: |
|
||||||||
Volume: | 7 | ||||||||
Article Number: | 25 | ||||||||
DOI: | 10.21203/rs.3.rs-3104286/v1 | ||||||||
Status: | Peer Reviewed | ||||||||
Publication Status: | Published | ||||||||
Access rights to Published version: | Open Access (Creative Commons) | ||||||||
Date of first compliant deposit: | 6 December 2023 | ||||||||
Date of first compliant Open Access: | 6 December 2023 | ||||||||
RIOXX Funder/Project Grant: |
|
||||||||
Open Access Version: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year