The Library
Why rankings of biomedical image analysis competitions should be interpreted with care
Tools
Tools
Tools
(2018) Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications, 9 (1). 5217. doi:10.1038/s41467-018-07619-7
|
PDF
WRAP-why-rankings-biomedical-image-interpreted-care-Rajpoot-2018.pdf - Published Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (813Kb) | Preview |
Official URL: http://dx.doi.org/10.1038/s41467-018-07619-7
Abstract
International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the importance of challenges and show that the lack of quality control has critical consequences. First, reproducibility and interpretation of the results is often hampered as only a fraction of relevant information is typically provided. Second, the rank of an algorithm is generally not robust to a number of variables such as the test data used for validation, the ranking scheme applied and the observers that make the reference annotations. To overcome these problems, we recommend best practice guidelines and define open research questions to be addressed in the future.
| Item Type: | Journal Article | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Subjects: | R Medicine > RC Internal medicine | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Divisions: | Faculty of Science > Computer Science | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Library of Congress Subject Headings (LCSH): | Diagnostic imaging, Imaging systems in medicine, Diagnostic imaging -- Competitions -- Evaluation | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Journal or Publication Title: | Nature Communications | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Publisher: | Nature Publishing Group | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ISSN: | 2041-1723 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Official Date: | 6 December 2018 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dates: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Volume: | 9 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Number: | 1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Article Number: | 5217 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| DOI: | 10.1038/s41467-018-07619-7 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Status: | Peer Reviewed | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Publication Status: | Published | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Access rights to Published version: | Open Access | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| RIOXX Funder/Project Grant: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
