Medical image synthesis with context-aware generative adversarial networks

Dong Nie, Roger Trullo, Jun Lian, Caroline Petitjean, Su Ruan, Qian Wang, Dinggang Shen

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    511 Citations (Scopus)

    Abstract

    Computed tomography (CT) is critical for various clinical applications, e.g., radiation treatment planning and also PET attenuation correction in MRI/PET scanner. However, CT exposes radiation during acquisition, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve radiations. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiation planning. In this paper, we propose a data-driven approach to address this challenging problem. Specifically, we train a fully convolutional network (FCN) to generate CT given the MR image. To better model the nonlinear mapping from MRI to CT and produce more realistic images, we propose to use the adversarial training strategy to train the FCN. Moreover, we propose an image-gradient-difference based loss function to alleviate the blurriness of the generated CT. We further apply Auto-Context Model (ACM) to implement a context-aware generative adversarial network. Experimental results show that our method is accurate and robust for predicting CT images from MR images, and also outperforms three state-of-the-art methods under comparison.

    Original languageEnglish
    Title of host publicationMedical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings
    EditorsLena Maier-Hein, Alfred Franz, Pierre Jannin, Simon Duchesne, Maxime Descoteaux, D. Louis Collins
    PublisherSpringer Verlag
    Pages417-425
    Number of pages9
    ISBN (Print)9783319661780
    DOIs
    Publication statusPublished - 2017
    Event20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017 - Quebec City, Canada
    Duration: 2017 Sept 112017 Sept 13

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume10435 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Other

    Other20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017
    Country/TerritoryCanada
    CityQuebec City
    Period17/9/1117/9/13

    Bibliographical note

    Publisher Copyright:
    © Springer International Publishing AG 2017.

    Keywords

    • Auto-context
    • Deep learning
    • GAN
    • Generative models
    • Image synthesis

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • General Computer Science

    Fingerprint

    Dive into the research topics of 'Medical image synthesis with context-aware generative adversarial networks'. Together they form a unique fingerprint.

    Cite this