3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation

Dong Nie, Li Wang, Ehsan Adeli, Cuijin Lao, Weili Lin, Dinggang Shen

Research output: Contribution to journalArticlepeer-review

128 Citations (Scopus)


Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.

Original languageEnglish
Article number8287819
Pages (from-to)1123-1136
Number of pages14
JournalIEEE Transactions on Cybernetics
Issue number3
Publication statusPublished - 2019 Mar

Bibliographical note

Publisher Copyright:
© 2013 IEEE.


  • 3-D fully convolutional network (3D-FCN)
  • brain MR image
  • isointense phase
  • multimodality MR images
  • tissue segmentation

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of '3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation'. Together they form a unique fingerprint.

Cite this