Visual speech recognition using weighted dynamic time warping

Kyungsun Lee, Minseok Keum, David K. Han, Hanseok Ko

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


It is unclear whether Hidden Markov Model (HMM) or Dynamic Time Warping (DTW) mapping is more appropriate for visual speech recognition when only small data samples are available. In this letter, the two approaches are compared in terms of sensitivity to the amount of training samples and computing time with the objective of determining the tipping point. The limited training data problem is addressed by exploiting a straightforward template matching via weighted-DTW. The proposed framework is a refined DTW by adjusting the warping paths with judicially injected weights to ensure a smooth diagonal path for accurate alignment without added computational load. The proposed WDTW is evaluated on three databases (two in the public domain and one developed in-house) for visual recognition performance. Subsequent experiments indicate that the proposed WDTW significantly enhances the recognition rate compared to the DTW and HMM based algorithms, especially under limited data samples.

Original languageEnglish
Pages (from-to)1430-1433
Number of pages4
JournalIEICE Transactions on Information and Systems
Issue number7
Publication statusPublished - 2015 Jul 1

Bibliographical note

Publisher Copyright:
Copyright © 2015 The Institute of Electronics, Information and Communication Engineers.


  • Lip reading
  • Visual speech recognition

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence


Dive into the research topics of 'Visual speech recognition using weighted dynamic time warping'. Together they form a unique fingerprint.

Cite this