The Library
Knowledge distillation in histology landscape by multi-layer features supervision
Tools
Javed, Sajid, Mahmood, Arif, Qaiser, Talha and Werghi, Naoufel (2023) Knowledge distillation in histology landscape by multi-layer features supervision. IEEE Journal of Biomedical and Health Informatics, 27 (4). pp. 2037-2046. doi:10.1109/jbhi.2023.3237749 ISSN 2168-2194.
|
PDF
WRAP-knowledge-distillation-histology-landscape-multi-layer-features-supervision-2023.pdf - Accepted Version - Requires a PDF viewer. Available under License Creative Commons Attribution 4.0. Download (3536Kb) | Preview |
Official URL: https://doi.org/10.1109/jbhi.2023.3237749
Abstract
Automatic tissue classification is a fundamental task in computational pathology for profiling tumor micro-environments. Deep learning has advanced tissue classification performance at the cost of significant computational power. Shallow networks have also been end-to-end trained using direct supervision however their performance degrades because of the lack of capturing robust tissue heterogeneity. Knowledge distillation has recently been employed to improve the performance of the shallow networks used as student networks by using additional supervision from deep neural networks used as teacher networks. In the current work, we propose a novel knowledge distillation algorithm to improve the performance of shallow networks for tissue phenotyping in histology images. For this purpose, we propose multi-layer feature distillation such that a single layer in the student network gets supervision from multiple teacher layers. In the proposed algorithm, the size of the feature map of two layers is matched by using a learnable multi-layer perceptron. The distance between the feature maps of the two layers is then minimized during the training of the student network. The overall objective function is computed by summation of the loss over multiple layers combination weighted with a learnable attention-based parameter. The proposed algorithm is named as Knowledge Distillation for Tissue Phenotyping (KDTP). Experiments are performed on five different publicly available histology image classification datasets using several teacher-student network combinations within the KDTP algorithm. Our results demonstrate a significant performance increase in the student networks by using the proposed KDTP algorithm compared to direct supervision-based training methods.
Item Type: | Journal Article | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Subjects: | R Medicine > RC Internal medicine | |||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | |||||||||
SWORD Depositor: | Library Publications Router | |||||||||
Library of Congress Subject Headings (LCSH): | Diagnostic imaging -- Data processing, Health informatics, Knowledge management -- Research, Histology -- Technique, Pattern recognition systems, Phenotype -- Research, Pathology -- Data processing | |||||||||
Journal or Publication Title: | IEEE Journal of Biomedical and Health Informatics | |||||||||
Publisher: | IEEE | |||||||||
ISSN: | 2168-2194 | |||||||||
Official Date: | April 2023 | |||||||||
Dates: |
|
|||||||||
Volume: | 27 | |||||||||
Number: | 4 | |||||||||
Page Range: | pp. 2037-2046 | |||||||||
DOI: | 10.1109/jbhi.2023.3237749 | |||||||||
Status: | Peer Reviewed | |||||||||
Publication Status: | Published | |||||||||
Access rights to Published version: | Open Access (Creative Commons) | |||||||||
Date of first compliant deposit: | 14 March 2023 | |||||||||
Date of first compliant Open Access: | 14 March 2023 | |||||||||
RIOXX Funder/Project Grant: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year