Difficulty-aware hierarchical convolutional neural networks for deformable registration of brain MR images

Yunzhi Huang, Sahar Ahmad, Jingfan Fan, Dinggang Shen, Pew Thian Yap

Research output: Contribution to journalArticlepeer-review

22 Citations (Scopus)


The aim of deformable brain image registration is to align anatomical structures, which can potentially vary with large and complex deformations. Anatomical structures vary in size and shape, requiring the registration algorithm to estimate deformation fields at various degrees of complexity. Here, we present a difficulty-aware model based on an attention mechanism to automatically identify hard-to-register regions, allowing better estimation of large complex deformations. The difficulty-aware model is incorporated into a cascaded neural network consisting of three sub-networks to fully leverage both global and local contextual information for effective registration. The first sub-network is trained at the image level to predict a coarse-scale deformation field, which is then used for initializing the subsequent sub-network. The next two sub-networks progressively optimize at the patch level with different resolutions to predict a fine-scale deformation field. Embedding difficulty-aware learning into the hierarchical neural network allows harder patches to be identified in the deeper sub-networks at higher resolutions for refining the deformation field. Experiments conducted on four public datasets validate that our method achieves promising registration accuracy with better preservation of topology, compared with state-of-the-art registration methods.

Original languageEnglish
Article number101817
JournalMedical Image Analysis
Publication statusPublished - 2021 Jan

Bibliographical note

Publisher Copyright:
© 2020 Elsevier B.V.


  • Brain MRI
  • Cascaded neural network
  • Deformable registration
  • Difficulty-aware sampling

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Difficulty-aware hierarchical convolutional neural networks for deformable registration of brain MR images'. Together they form a unique fingerprint.

Cite this