Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation

Xuhua Ren, Sahar Ahmad, Lichi Zhang, Lei Xiang, Dong Nie, Fan Yang, Qian Wang, DInggang Shen

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)


Semantic segmentation is essentially important to biomedical image analysis. Many recent works mainly focus on integrating the Fully Convolutional Network (FCN) architecture with sophisticated convolution implementation and deep supervision. Such complex networks need large training datasets, a requirement which is challenging for medical image analysis. In this paper, we propose to decompose the single segmentation task into three subsequent sub-tasks, including (1) pixel-wise image semantic segmentation, (2) prediction of the instance class labels of the objects within the image, and (3) classification of the scene the image belonging to. While these three sub-tasks are trained to optimize their individual loss functions at different perceptual levels, we propose to allow their interaction within the task-task context ensemble. Moreover, we propose a novel sync-regularization to penalize the deviation between the outputs of the pixel-wise semantic segmentation and the instance class prediction tasks. These effective regularizations help FCN utilize context information comprehensively and attain accurate segmentation, even though the number of images for training may be limited in many biomedical applications. We have successfully applied our framework to three diverse 2D/3D medical image datasets, including Robotic Scene Segmentation Challenge 18 (ROBOT18), Brain Tumor Segmentation Challenge 18 (BRATS18), and Retinal Fundus Glaucoma Challenge (REFUGE18). We have achieved outperformed or comparable performance in all the three challenges. Our code, typical data and trained models are available at https://github.com/xuhuaren/TDSNet.

Original languageEnglish
Article number9126262
Pages (from-to)7497-7510
Number of pages14
JournalIEEE Transactions on Image Processing
Publication statusPublished - 2020

Bibliographical note

Publisher Copyright:
© 1992-2012 IEEE.


  • Semantic segmentation
  • deep learning
  • fully convolutional network
  • sync-regularization
  • task decomposition

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation'. Together they form a unique fingerprint.

Cite this