3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation

Dong Nie, Li Wang, Ehsan Adeli, Cuijin Lao, Weili Lin, Dinggang Shen

    Research output: Contribution to journalArticlepeer-review

    146 Citations (Scopus)

    Abstract

    Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.

    Original languageEnglish
    Article number8287819
    Pages (from-to)1123-1136
    Number of pages14
    JournalIEEE Transactions on Cybernetics
    Volume49
    Issue number3
    DOIs
    Publication statusPublished - 2019 Mar

    Bibliographical note

    Publisher Copyright:
    © 2013 IEEE.

    Keywords

    • 3-D fully convolutional network (3D-FCN)
    • brain MR image
    • isointense phase
    • multimodality MR images
    • tissue segmentation

    ASJC Scopus subject areas

    • Software
    • Control and Systems Engineering
    • Information Systems
    • Human-Computer Interaction
    • Computer Science Applications
    • Electrical and Electronic Engineering

    Fingerprint

    Dive into the research topics of '3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation'. Together they form a unique fingerprint.

    Cite this