Modality classification and concept detection in medical images using deep transfer learning

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

Abstract

Medical image classification and concept detection are two important tasks for efficient and robust medical retrieval systems and also help with downstream tasks such as knowledge discovery, medical report generation, medical question answering, and clinical decision making. We investigate the effectiveness of transfer learning on the modality classification task using state-of-the-art deep convolutional neural networks pretrained on generic images. We also compare the performance of the traditional pipeline of handcrafted features with multi-label learning algorithms with end-to-end deep learning features for the concept detection task. Experimental results on the modality classification task show that transfer learning can leverage the patterns learned from large training data to the medical domain where little labeled data is available. Moreover, results on the concept detection task show that the deep learning approach provides better and more powerful feature representations compared to handcrafted feature extraction methods. The results on both tasks suggest that deep transfer learning methods are effective in the medical domain where data is scarce.

LanguageEnglish
Title of host publication2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1-9
Number of pages9
Volume2018-November
ISBN (Electronic)9781728101255
ISBN (Print)9781728101262
DOIs
Publication statusPublished - 2018
Event2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018 - Auckland, New Zealand
Duration: 19 Nov 201821 Nov 2018

Conference

Conference2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018
CountryNew Zealand
CityAuckland
Period19/11/1821/11/18

Fingerprint

Image classification
Learning algorithms
Data mining
Feature extraction
Labels
Pipelines
Decision making
Neural networks
Deep learning

Keywords

  • Concept Detection
  • Convolutional Neural Networks
  • Deep Learning
  • Medical Image Analysis
  • Medical Image Classification
  • Modality Classification
  • Multi-Label learning

Cite this

Singh, S., Ho-Shon, K., Karimi, S., & Hamey, L. (2018). Modality classification and concept detection in medical images using deep transfer learning. In 2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018 (Vol. 2018-November, pp. 1-9). [8634803] Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/IVCNZ.2018.8634803
Singh, Sonit ; Ho-Shon, Kevin ; Karimi, Sarvnaz ; Hamey, Len. / Modality classification and concept detection in medical images using deep transfer learning. 2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018. Vol. 2018-November Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE), 2018. pp. 1-9
@inproceedings{8622e07039444b079bd81c33f3e66809,
title = "Modality classification and concept detection in medical images using deep transfer learning",
abstract = "Medical image classification and concept detection are two important tasks for efficient and robust medical retrieval systems and also help with downstream tasks such as knowledge discovery, medical report generation, medical question answering, and clinical decision making. We investigate the effectiveness of transfer learning on the modality classification task using state-of-the-art deep convolutional neural networks pretrained on generic images. We also compare the performance of the traditional pipeline of handcrafted features with multi-label learning algorithms with end-to-end deep learning features for the concept detection task. Experimental results on the modality classification task show that transfer learning can leverage the patterns learned from large training data to the medical domain where little labeled data is available. Moreover, results on the concept detection task show that the deep learning approach provides better and more powerful feature representations compared to handcrafted feature extraction methods. The results on both tasks suggest that deep transfer learning methods are effective in the medical domain where data is scarce.",
keywords = "Concept Detection, Convolutional Neural Networks, Deep Learning, Medical Image Analysis, Medical Image Classification, Modality Classification, Multi-Label learning",
author = "Sonit Singh and Kevin Ho-Shon and Sarvnaz Karimi and Len Hamey",
year = "2018",
doi = "10.1109/IVCNZ.2018.8634803",
language = "English",
isbn = "9781728101262",
volume = "2018-November",
pages = "1--9",
booktitle = "2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018",
publisher = "Institute of Electrical and Electronics Engineers (IEEE)",
address = "United States",

}

Singh, S, Ho-Shon, K, Karimi, S & Hamey, L 2018, Modality classification and concept detection in medical images using deep transfer learning. in 2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018. vol. 2018-November, 8634803, Institute of Electrical and Electronics Engineers (IEEE), Piscataway, NJ, pp. 1-9, 2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018, Auckland, New Zealand, 19/11/18. https://doi.org/10.1109/IVCNZ.2018.8634803

Modality classification and concept detection in medical images using deep transfer learning. / Singh, Sonit; Ho-Shon, Kevin; Karimi, Sarvnaz; Hamey, Len.

2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018. Vol. 2018-November Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE), 2018. p. 1-9 8634803.

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionResearchpeer-review

TY - GEN

T1 - Modality classification and concept detection in medical images using deep transfer learning

AU - Singh, Sonit

AU - Ho-Shon, Kevin

AU - Karimi, Sarvnaz

AU - Hamey, Len

PY - 2018

Y1 - 2018

N2 - Medical image classification and concept detection are two important tasks for efficient and robust medical retrieval systems and also help with downstream tasks such as knowledge discovery, medical report generation, medical question answering, and clinical decision making. We investigate the effectiveness of transfer learning on the modality classification task using state-of-the-art deep convolutional neural networks pretrained on generic images. We also compare the performance of the traditional pipeline of handcrafted features with multi-label learning algorithms with end-to-end deep learning features for the concept detection task. Experimental results on the modality classification task show that transfer learning can leverage the patterns learned from large training data to the medical domain where little labeled data is available. Moreover, results on the concept detection task show that the deep learning approach provides better and more powerful feature representations compared to handcrafted feature extraction methods. The results on both tasks suggest that deep transfer learning methods are effective in the medical domain where data is scarce.

AB - Medical image classification and concept detection are two important tasks for efficient and robust medical retrieval systems and also help with downstream tasks such as knowledge discovery, medical report generation, medical question answering, and clinical decision making. We investigate the effectiveness of transfer learning on the modality classification task using state-of-the-art deep convolutional neural networks pretrained on generic images. We also compare the performance of the traditional pipeline of handcrafted features with multi-label learning algorithms with end-to-end deep learning features for the concept detection task. Experimental results on the modality classification task show that transfer learning can leverage the patterns learned from large training data to the medical domain where little labeled data is available. Moreover, results on the concept detection task show that the deep learning approach provides better and more powerful feature representations compared to handcrafted feature extraction methods. The results on both tasks suggest that deep transfer learning methods are effective in the medical domain where data is scarce.

KW - Concept Detection

KW - Convolutional Neural Networks

KW - Deep Learning

KW - Medical Image Analysis

KW - Medical Image Classification

KW - Modality Classification

KW - Multi-Label learning

UR - http://www.scopus.com/inward/record.url?scp=85062770991&partnerID=8YFLogxK

U2 - 10.1109/IVCNZ.2018.8634803

DO - 10.1109/IVCNZ.2018.8634803

M3 - Conference proceeding contribution

SN - 9781728101262

VL - 2018-November

SP - 1

EP - 9

BT - 2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018

PB - Institute of Electrical and Electronics Engineers (IEEE)

CY - Piscataway, NJ

ER -

Singh S, Ho-Shon K, Karimi S, Hamey L. Modality classification and concept detection in medical images using deep transfer learning. In 2018 International Conference on Image and Vision Computing New Zealand, IVCNZ 2018. Vol. 2018-November. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE). 2018. p. 1-9. 8634803 https://doi.org/10.1109/IVCNZ.2018.8634803