Viseme recognition experiment using context dependent hidden Markov models

Soonkyu Lee; Dongsuk Yook

doi:10.1007/3-540-45675-9_84

Viseme recognition experiment using context dependent hidden Markov models

Soonkyu Lee
, Dongsuk Yook

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

Visual images synchronized with audio signals can provide user-friendly interface for man machine interactions. The visual speech can be represented as a sequence of visemes, which are the generic face images corresponding to particular sounds. We use HMMs (hidden Markov models) to convert audio signals to a sequence of visemes. In this paper, we compare two approaches in using HMMs. In the first approach, an HMM is trained for each triviseme which is a viseme with its left and right context, and the audio signals are directly recognized as a sequence of trivisemes. In the second approach, each triphone is modeled with an HMM, and a general triphone recognizer is used to produce a triphone sequence from the audio signals. The triviseme or triphone sequence is then converted to a viseme sequence. The performances of the two viseme recognition systems are evaluated on the TIMIT speech corpus.

Original language	English
Title of host publication	Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings
Editors	Hujun Yin, Nigel Allinson, Richard Freeman, John Keane, Simon Hubbard
Publisher	Springer Verlag
Pages	557-561
Number of pages	5
ISBN (Print)	9783540440253
DOIs	https://doi.org/10.1007/3-540-45675-9_84
Publication status	Published - 2002
Event	3rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002 - Manchester, United Kingdom Duration: 2002 Aug 12 → 2002 Aug 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	2412
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	3rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002
Country/Territory	United Kingdom
City	Manchester
Period	02/8/12 → 02/8/14

Bibliographical note

Publisher Copyright:
© Springer-Verlag Berlin Heidelberg 2002.

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/3-540-45675-9_84

Cite this

Lee, S., & Yook, D. (2002). Viseme recognition experiment using context dependent hidden Markov models. In H. Yin, N. Allinson, R. Freeman, J. Keane, & S. Hubbard (Eds.), Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings (pp. 557-561). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2412). Springer Verlag. https://doi.org/10.1007/3-540-45675-9_84

Viseme recognition experiment using context dependent hidden Markov models. / Lee, Soonkyu; Yook, Dongsuk.
Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings. ed. / Hujun Yin; Nigel Allinson; Richard Freeman; John Keane; Simon Hubbard. Springer Verlag, 2002. p. 557-561 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2412).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lee, S & Yook, D 2002, Viseme recognition experiment using context dependent hidden Markov models. in H Yin, N Allinson, R Freeman, J Keane & S Hubbard (eds), Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2412, Springer Verlag, pp. 557-561, 3rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002, Manchester, United Kingdom, 02/8/12. https://doi.org/10.1007/3-540-45675-9_84

Lee S, Yook D. Viseme recognition experiment using context dependent hidden Markov models. In Yin H, Allinson N, Freeman R, Keane J, Hubbard S, editors, Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings. Springer Verlag. 2002. p. 557-561. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/3-540-45675-9_84

Lee, Soonkyu ; Yook, Dongsuk. / Viseme recognition experiment using context dependent hidden Markov models. Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings. editor / Hujun Yin ; Nigel Allinson ; Richard Freeman ; John Keane ; Simon Hubbard. Springer Verlag, 2002. pp. 557-561 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{e383ad51b6d843a4aa963e7a80904188,

title = "Viseme recognition experiment using context dependent hidden Markov models",

abstract = "Visual images synchronized with audio signals can provide user-friendly interface for man machine interactions. The visual speech can be represented as a sequence of visemes, which are the generic face images corresponding to particular sounds. We use HMMs (hidden Markov models) to convert audio signals to a sequence of visemes. In this paper, we compare two approaches in using HMMs. In the first approach, an HMM is trained for each triviseme which is a viseme with its left and right context, and the audio signals are directly recognized as a sequence of trivisemes. In the second approach, each triphone is modeled with an HMM, and a general triphone recognizer is used to produce a triphone sequence from the audio signals. The triviseme or triphone sequence is then converted to a viseme sequence. The performances of the two viseme recognition systems are evaluated on the TIMIT speech corpus.",

author = "Soonkyu Lee and Dongsuk Yook",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2002.; 3rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002 ; Conference date: 12-08-2002 Through 14-08-2002",

year = "2002",

doi = "10.1007/3-540-45675-9\_84",

language = "English",

isbn = "9783540440253",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "557--561",

editor = "Hujun Yin and Nigel Allinson and Richard Freeman and John Keane and Simon Hubbard",

booktitle = "Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings",

}

TY - GEN

T1 - Viseme recognition experiment using context dependent hidden Markov models

AU - Lee, Soonkyu

AU - Yook, Dongsuk

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2002.

PY - 2002

Y1 - 2002

N2 - Visual images synchronized with audio signals can provide user-friendly interface for man machine interactions. The visual speech can be represented as a sequence of visemes, which are the generic face images corresponding to particular sounds. We use HMMs (hidden Markov models) to convert audio signals to a sequence of visemes. In this paper, we compare two approaches in using HMMs. In the first approach, an HMM is trained for each triviseme which is a viseme with its left and right context, and the audio signals are directly recognized as a sequence of trivisemes. In the second approach, each triphone is modeled with an HMM, and a general triphone recognizer is used to produce a triphone sequence from the audio signals. The triviseme or triphone sequence is then converted to a viseme sequence. The performances of the two viseme recognition systems are evaluated on the TIMIT speech corpus.

AB - Visual images synchronized with audio signals can provide user-friendly interface for man machine interactions. The visual speech can be represented as a sequence of visemes, which are the generic face images corresponding to particular sounds. We use HMMs (hidden Markov models) to convert audio signals to a sequence of visemes. In this paper, we compare two approaches in using HMMs. In the first approach, an HMM is trained for each triviseme which is a viseme with its left and right context, and the audio signals are directly recognized as a sequence of trivisemes. In the second approach, each triphone is modeled with an HMM, and a general triphone recognizer is used to produce a triphone sequence from the audio signals. The triviseme or triphone sequence is then converted to a viseme sequence. The performances of the two viseme recognition systems are evaluated on the TIMIT speech corpus.

UR - https://www.scopus.com/pages/publications/84947935184

U2 - 10.1007/3-540-45675-9_84

DO - 10.1007/3-540-45675-9_84

M3 - Conference contribution

AN - SCOPUS:84947935184

SN - 9783540440253

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 557

EP - 561

BT - Intelligent Data Engineering and Automated Learning - IDEAL 2002 - 3rd International Conference, Proceedings

A2 - Yin, Hujun

A2 - Allinson, Nigel

A2 - Freeman, Richard

A2 - Keane, John

A2 - Hubbard, Simon

PB - Springer Verlag

T2 - 3rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2002

Y2 - 12 August 2002 through 14 August 2002

ER -

Viseme recognition experiment using context dependent hidden Markov models

Abstract

Publication series

Other

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this