MUSEMBLE: A novel music retrieval system with automatic voice query transcription and reformulation

Seungmin Rho, Byeong jun Han, Eenjun Hwang, Minkoo Kim

Research output: Contribution to journalArticlepeer-review

26 Citations (Scopus)


So far, many researches have been done to develop efficient music retrieval systems, and query-by-humming has been considered as one of the most intuitive and effective query methods for music retrieval. For the voice humming to be a reliable query source, elaborate signal processing and acoustic similarity measurement schemes are necessary. On the other hand, recently, there has been an increased interest in query reformulation using relevance feedback with evolutionary techniques such as genetic algorithm for multimedia information retrieval. However, these techniques have not been exploited widely in the field of music retrieval. In this paper, we develop a novel music retrieval system called MUSEMBLE (MUSic enEMBLE) based on two distinct features: (i) A sung or hummed query is automatically transcribed into a sequence of pitch and duration pairs with improved accuracy for music representation. More specifically, we developed two new and unique techniques called WAE (windowed average energy) and dynamic ADF (amplitude-based difference function) onsets for more accurate note segmentation and onset/offset detection in acoustic signal, respectively. The former improved energy-based approaches such as AE by defining small but coherent windows with local and global threshold values. On the other hand, the latter improved the AF (amplitude function) that calculates the summation of the absolute values of signal differences for the clustering energy contour. (ii) A user query is reformulated using user relevance feedback with a genetic algorithm to improve retrieval performance. Even though we have especially focused on humming queries in this paper, MUSEMBLE provides versatile query and browsing interfaces for various kinds of users. We have carried out extensive experiments on the prototype system to evaluate the performance of our voice query transcription and genetic algorithm-based relevance feedback schemes. We demonstrate that our proposed method improves the retrieval accuracy up to 20-40% compared with other popular RF methods. We also show that both WAE and Dynamic ADF methods improve the transcription accuracy up to 95%.

Original languageEnglish
Pages (from-to)1065-1080
Number of pages16
JournalJournal of Systems and Software
Issue number7
Publication statusPublished - 2008 Jul

Bibliographical note

Funding Information:
This research was supported by the MIC (Ministry of Information and Communication), Korea, under the ITRC (Information Technology Research Center) support program supervised by the IITA (Institute of Information Technology Advancement). (IITA-2006-(C1090-0603-0002)) and this research is also supported by the Ubiquitous Computing and Network (UCN) Project, the Ministry of Information and Communication (MIC) 21st Century Frontier R&D Program in Korea.


  • Genetic algorithm
  • Multimedia database
  • Music retrieval
  • Pitch tracking
  • Relevance feedback
  • Signal processing

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture


Dive into the research topics of 'MUSEMBLE: A novel music retrieval system with automatic voice query transcription and reformulation'. Together they form a unique fingerprint.

Cite this