The Library
Avoiding C-hacking when evaluating survival distribution predictions with discrimination measures
Tools
Tools
Tools
Sonabend, Raphael, Bender, Andreas and Vollmer, Sebastian (2022) Avoiding C-hacking when evaluating survival distribution predictions with discrimination measures. Bioinformatics, 38 (17). pp. 4178-4184. doi:10.1093/bioinformatics/btac451 ISSN 1460-2059.
|
PDF
WRAP-Avoiding-C-hacking-evaluating-survival-predictions-discrimination-measures-22.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (499Kb) | Preview |
Official URL: https://doi.org/10.1093/bioinformatics/btac451
Abstract
Motivation In this paper we consider how to evaluate survival distribution predictions with measures of discrimination. This is non-trivial as discrimination measures are the most commonly used in survival analysis and yet there is no clear method to derive a risk prediction from a distribution prediction. We survey methods proposed in literature and software and consider their respective advantages and disadvantages. Results Whilst distributions are frequently evaluated by discrimination measures, we find that the method for doing so is rarely described in the literature and often leads to unfair comparisons or ‘C-hacking’. We demonstrate by example how simple it can be to manipulate results and use this to argue for better reporting guidelines and transparency in the literature. We recommend that machine learning survival analysis software implements clear transformations between distribution and risk predictions in order to allow more transparent and accessible model evaluation. Availability The code used in the final experiment is available at https://github.com/RaphaelS1/distribution_discrimination. Supplementary information Supplementary data are available at Bioinformatics online.
| Item Type: | Journal Article | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Divisions: | Faculty of Science, Engineering and Medicine > Science > Mathematics | ||||||||
| SWORD Depositor: | Library Publications Router | ||||||||
| Journal or Publication Title: | Bioinformatics | ||||||||
| Publisher: | Oxford University Press (OUP) | ||||||||
| ISSN: | 1460-2059 | ||||||||
| Official Date: | September 2022 | ||||||||
| Dates: |
|
||||||||
| Volume: | 38 | ||||||||
| Number: | 17 | ||||||||
| Page Range: | pp. 4178-4184 | ||||||||
| DOI: | 10.1093/bioinformatics/btac451 | ||||||||
| Status: | Peer Reviewed | ||||||||
| Publication Status: | Published | ||||||||
| Reuse Statement (publisher, data, author rights): | ** Article version: AM ** From Crossref journal articles via Jisc Publications Router ** History: epub 12-07-2022; issued 12-07-2022. ** Licence for AM version of this article starting on 12-07-2022: https://creativecommons.org/licenses/by/4.0/ | ||||||||
| Access rights to Published version: | Open Access (Creative Commons) | ||||||||
| Date of first compliant deposit: | 21 September 2022 | ||||||||
| Date of first compliant Open Access: | 21 September 2022 | ||||||||
| Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
