Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person

Seongjae Lee, Sunmee Kang, David K. Han, Hanseok Ko

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


A novel approach for assisting bidirectional communication between people of normal hearing and hearing-impaired is presented. While the existing hearing-impaired assistive devices such as hearing aids and cochlear implants are vulnerable in extreme noise conditions or post-surgery side effects, the proposed concept is an alternative approach wherein spoken dialogue is achieved by means of employing a robust speech recognition technique which takes into consideration of noisy environmental factors without any attachment into human body. The proposed system is a portable device with an acoustic beamformer for directional noise reduction and capable of performing speech-to-text transcription function, which adopts a keyword spotting method. It is also equipped with an optimized user interface for hearing-impaired people, rendering intuitive and natural device usage with diverse domain contexts. The relevant experimental results confirm that the proposed interface design is feasible for realizing an effective and efficient intelligent agent for hearing-impaired.

Original languageEnglish
Pages (from-to)915-926
Number of pages12
JournalMedical and Biological Engineering and Computing
Issue number6
Publication statusPublished - 2016 Jun 1

Bibliographical note

Funding Information:
This research was supported by Ministry of Health & Welfare R&D (A111189) and partially supported by Seokyeong University grant programme in 2013.

Publisher Copyright:
© 2016, International Federation for Medical and Biological Engineering.


  • Beamforming
  • Hearing-impaired person
  • Keyword spotting
  • Speech-to-text transcription
  • User interface

ASJC Scopus subject areas

  • Biomedical Engineering
  • Computer Science Applications


Dive into the research topics of 'Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person'. Together they form a unique fingerprint.

Cite this