Deep Bayesian Hashing with Center Prior for Multi-Modal Neuroimage Retrieval

Erkun Yang, Mingxia Liu, Dongren Yao, Bing Cao, Chunfeng Lian, Pew Thian Yap, Dinggang Shen

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


Multi-modal neuroimage retrieval has greatly facilitated the efficiency and accuracy of decision making in clinical practice by providing physicians with previous cases (with visually similar neuroimages) and corresponding treatment records. However, existing methods for image retrieval usually fail when applied directly to multi-modal neuroimage databases, since neuroimages generally have smaller inter-class variation and larger inter-modal discrepancy compared to natural images. To this end, we propose a deep Bayesian hash learning framework, called CenterHash, which can map multi-modal data into a shared Hamming space and learn discriminative hash codes from imbalanced multi-modal neuroimages. The key idea to tackle the small inter-class variation and large inter-modal discrepancy is to learn a common center representation for similar neuroimages from different modalities and encourage hash codes to be explicitly close to their corresponding center representations. Specifically, we measure the similarity between hash codes and their corresponding center representations and treat it as a center prior in the proposed Bayesian learning framework. A weighted contrastive likelihood loss function is also developed to facilitate hash learning from imbalanced neuroimage pairs. Comprehensive empirical evidence shows that our method can generate effective hash codes and yield state-of-The-Art performance in cross-modal retrieval on three multi-modal neuroimage datasets.

Original languageEnglish
Article number9222290
Pages (from-to)503-513
Number of pages11
JournalIEEE Transactions on Medical Imaging
Issue number2
Publication statusPublished - 2021 Feb


  • Deep Bayesian hashing
  • MRI
  • PET
  • multi-modal neuroimage
  • retrieval

ASJC Scopus subject areas

  • Software
  • Radiological and Ultrasound Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Deep Bayesian Hashing with Center Prior for Multi-Modal Neuroimage Retrieval'. Together they form a unique fingerprint.

Cite this