MS-GAN: GAN-based semantic segmentation of multiple sclerosis lesions in brain magnetic resonance imaging

Chaoyi Zhang, Yang Song, Sidong Liu, Scott Lill, Chenyu Wang, Zihao Tang, Yuyi You, Yang Gao, Alexander Klistorner, Michael Barnett, Weidong Cai

    Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

    14 Citations (Scopus)

    Abstract

    Automated segmentation of multiple sclerosis (MS) lesions in brain imaging is challenging due to the high variability in lesion characteristics. Based on the generative adversarial network (GAN), we propose a semantic segmentation framework MS-GAN to localize MS lesions in multimodal brain magnetic resonance imaging (MRI), which consists of one multimodal encoder-decoder generator G and multiple discriminators D corresponding to the multiple input modalities. For the design of the generator, we adopt an encoder-decoder deep learning architecture with bypass of spatial information from encoder to the corresponding decoder, which helps to reduce the network parameters while improving the localization performance. Our generator is also designed to integrate multimodal imaging data in end-to-end learning with multi-path encoding and cross-modality fusion. An additional classification-related constraint is proposed for the adversarial training process of the GAN model, with the aim of alleviating the hard-to-converge issue in classification-based image-to-image translation problems. For evaluation, we collected a database of 126 cases from patients with relapsing MS. We also experimented with other semantic segmentation models as well as patch-based deep learning methods for performance comparison. The results show that our method provides more accurate segmentation than the state-of-the-art techniques.

    Original languageEnglish
    Title of host publication2018 International Conference on Digital Image Computing
    Subtitle of host publicationTechniques and Applications, DICTA 2018
    EditorsManzur Murshed, Manoranjan Paul, Md Asikuzzaman, Mark Pickering, Ambarish Natu, Antonio Robles-Kelly, Shaodi You, Lihong Zheng, Ashfaqur Rahman
    Place of PublicationPiscataway, NJ
    PublisherInstitute of Electrical and Electronics Engineers (IEEE)
    Pages84-91
    Number of pages8
    ISBN (Electronic)9781538666029
    DOIs
    Publication statusPublished - 2018
    Event2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2018 - Canberra, Australia
    Duration: 10 Dec 201813 Dec 2018

    Conference

    Conference2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2018
    Country/TerritoryAustralia
    CityCanberra
    Period10/12/1813/12/18

    Keywords

    • GAN
    • Image-to-image Translation
    • Multiple Sclerosis
    • Semantic Segmentation

    Fingerprint

    Dive into the research topics of 'MS-GAN: GAN-based semantic segmentation of multiple sclerosis lesions in brain magnetic resonance imaging'. Together they form a unique fingerprint.

    Cite this