Medical image synthesis with context-aware generative adversarial networks

Dong Nie, Roger Trullo, Jun Lian, Caroline Petitjean, Su Ruan, Qian Wang, Dinggang Shen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

385 Citations (Scopus)


Computed tomography (CT) is critical for various clinical applications, e.g., radiation treatment planning and also PET attenuation correction in MRI/PET scanner. However, CT exposes radiation during acquisition, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve radiations. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiation planning. In this paper, we propose a data-driven approach to address this challenging problem. Specifically, we train a fully convolutional network (FCN) to generate CT given the MR image. To better model the nonlinear mapping from MRI to CT and produce more realistic images, we propose to use the adversarial training strategy to train the FCN. Moreover, we propose an image-gradient-difference based loss function to alleviate the blurriness of the generated CT. We further apply Auto-Context Model (ACM) to implement a context-aware generative adversarial network. Experimental results show that our method is accurate and robust for predicting CT images from MR images, and also outperforms three state-of-the-art methods under comparison.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings
EditorsLena Maier-Hein, Alfred Franz, Pierre Jannin, Simon Duchesne, Maxime Descoteaux, D. Louis Collins
PublisherSpringer Verlag
Number of pages9
ISBN (Print)9783319661780
Publication statusPublished - 2017
Externally publishedYes
Event20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017 - Quebec City, Canada
Duration: 2017 Sept 112017 Sept 13

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10435 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017
CityQuebec City


  • Auto-context
  • Deep learning
  • GAN
  • Generative models
  • Image synthesis

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Medical image synthesis with context-aware generative adversarial networks'. Together they form a unique fingerprint.

Cite this