Privacy-preserving time-series medical images analysis using a hybrid deep learning framework

Zijie Yue, Shuai Ding, Lei Zhao, Youtao Zhang, Zehong Cao, M. Tanveer, Alireza Jolfaei, Xi Zheng

Research output: Contribution to journalArticlepeer-review

17 Citations (Scopus)


Time-series medical images are an important type of medical data that contain rich temporal and spatial information. As a state-of-the-art, computer-aided diagnosis (CAD) algorithms are usually used on these image sequences to improve analysis accuracy. However, such CAD algorithms are often required to upload medical images to honest-but-curious servers, which introduces severe privacy concerns. To preserve privacy, the existing CAD algorithms support analysis on each encrypted image but not on the whole encrypted image sequences, which leads to the loss of important temporal information among frames. To meet this challenge, a convolutional-LSTM network, named HE-CLSTM, is proposed for analyzing time-series medical images encrypted by a fully homomorphic encryption mechanism. Specifically, several convolutional blocks are constructed to extract discriminative spatial features, and LSTM-based sequence analysis layers (HE-LSTM) are leveraged to encode temporal information from the encrypted image sequences. Moreover, a weighted unit and a sequence voting layer are designed to incorporate both spatial and temporal features with different weights to improve performance while reducing the missed diagnosis rate. The experimental results on two challenging benchmarks (a Cervigram dataset and the BreaKHis public dataset) provide strong evidence that our framework can encode visual representations and sequential dynamics from encrypted medical image sequences; our method achieved AUCs above 0.94 both on the Cervigram and BreaKHis datasets, constituting a significant margin of statistical improvement compared with several competing methods.
Original languageEnglish
Article number57
Pages (from-to)1-21
Number of pages21
JournalACM Transactions on Internet Technology
Issue number3
Early online date16 Jun 2021
Publication statusPublished - Aug 2021


  • Time-series medical images analysis
  • deep learning


Dive into the research topics of 'Privacy-preserving time-series medical images analysis using a hybrid deep learning framework'. Together they form a unique fingerprint.

Cite this