TY - JOUR
T1 - 3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation
AU - Nie, Dong
AU - Wang, Li
AU - Adeli, Ehsan
AU - Lao, Cuijin
AU - Lin, Weili
AU - Shen, Dinggang
N1 - Funding Information:
Manuscript received December 5, 2016; revised November 20, 2017, January 15, 2018, and January 18, 2018; accepted January 19, 2018. Date of publication February 8, 2018; date of current version February 14, 2019. This work was supported in part by the National Institutes of Health under Grant MH109773, Grant MH100217, Grant MH070890, Grant EB006733, Grant EB008374, Grant EB009634, Grant AG041721, Grant AG042599, and Grant MH088520, and in part by the NIH-supported National Database for Autism Research. This paper was recommended by Associate Editor M. Shin. (Corresponding author: Dinggang Shen.) D. Nie is with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27510 USA, and also with the Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27510 USA.
Publisher Copyright:
© 2013 IEEE.
PY - 2019/3
Y1 - 2019/3
N2 - Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.
AB - Accurate segmentation of infant brain images into different regions of interest is one of the most important fundamental steps in studying early brain development. In the isointense phase (approximately 6-8 months of age), white matter and gray matter exhibit similar levels of intensities in magnetic resonance (MR) images, due to the ongoing myelination and maturation. This results in extremely low tissue contrast and thus makes tissue segmentation very challenging. Existing methods for tissue segmentation in this isointense phase usually employ patch-based sparse labeling on single modality. To address the challenge, we propose a novel 3-D multimodal fully convolutional network (FCN) architecture for segmentation of isointense phase brain MR images. Specifically, we extend the conventional FCN architectures from 2-D to 3-D, and, rather than directly using FCN, we intuitively integrate coarse (naturally high-resolution) and dense (highly semantic) feature maps to better model tiny tissue regions, in addition, we further propose a transformation module to better connect the aggregating layers; we also propose a fusion module to better serve the fusion of feature maps. We compare the performance of our approach with several baseline and state-of-the-art methods on two sets of isointense phase brain images. The comparison results show that our proposed 3-D multimodal FCN model outperforms all previous methods by a large margin in terms of segmentation accuracy. In addition, the proposed framework also achieves faster segmentation results compared to all other methods. Our experiments further demonstrate that: 1) carefully integrating coarse and dense feature maps can considerably improve the segmentation performance; 2) batch normalization can speed up the convergence of the networks, especially when hierarchical feature aggregations occur; and 3) integrating multimodal information can further boost the segmentation performance.
KW - 3-D fully convolutional network (3D-FCN)
KW - brain MR image
KW - isointense phase
KW - multimodality MR images
KW - tissue segmentation
UR - http://www.scopus.com/inward/record.url?scp=85041812887&partnerID=8YFLogxK
U2 - 10.1109/TCYB.2018.2797905
DO - 10.1109/TCYB.2018.2797905
M3 - Article
C2 - 29994385
AN - SCOPUS:85041812887
SN - 2168-2267
VL - 49
SP - 1123
EP - 1136
JO - IEEE Transactions on Cybernetics
JF - IEEE Transactions on Cybernetics
IS - 3
M1 - 8287819
ER -