High-level feature based PET image retrieval with deep learning architecture

Siqi Liu, Sidong Liu, Weidong Cai, Hangyu Che, Sonia Pujol, Ron Kikinis, Michael Fulham, Dagan Feng

Research output: Contribution to journalMeeting abstractpeer-review


Objectives: To develop a novel framework for accurate content-based PET images retrieval that utilises high-level ROI features with a deep learning architecture.

Methods: We acquired 331 3D PET images from the Alzheimer’s disease Neuroimaging Initiative (ADNI) cohort, including 77 normal control, 102 non-convertible mild cognitive impairment (ncMCI), 67 convertible MCI (cMCI) and 85 Alzheimer’s disease patients. All the 3D PET images were converted to ADNI format following the ADNI correction protocol and registered to the ICBM_152 template with 83 ROIs. The average cerebral glucose metabolic rate (CMRGlc) was extracted as the initial feature and further filtered by a deep learning architecture which consisted of multi-layered stacked autoencoders and were trained with these pre-labelled samples. The extracted high-level feature parameters derived from deep learning were used as the encodings for content-based retrieval with the K-nearest-neighbour (KNN) algorithm. We evaluated our framework on its Mean Average Precision (MAP) using the leave-one-out paradigm. The number of relevant images in MAP was set to 5.

Results: The proposed retrieval framework outperformed the most widely used state-of-the-art data representation methods, i.e, ISOMAP (49.18%) and Elastic Net (51.35%) with an overall MAP of 56.13%. The MAP values of three individual labels were higher than the other methods, except cMCI which has fewest samples in the dataset.

Conclusions: The high-level PET ROI feature extracted by deep learning can improve the accuracy of PET retrieval, comparing to the conventional data representation algorithms. This design of PET image retrieval framework with deep learning architecture offers potential advantages in medical image data management as well as statistical and comparative analyses of functional image data.

Research Support: ARC, AADRF, NA-MIC (NIH U54EB005149) and NAC (NIH P41EB015902)
Original languageEnglish
Article number2028
Number of pages1
JournalThe Journal of Nuclear Medicine
Issue numberSupplement 1
Publication statusPublished - 18 Aug 2014
Externally publishedYes


Dive into the research topics of 'High-level feature based PET image retrieval with deep learning architecture'. Together they form a unique fingerprint.

Cite this