Articulatory information for noise robust speech recognition

Vikramjit Mitra; Hosung Nam; Carol Y. Espy-Wilson; Elliot Saltzman; Louis Goldstein

doi:10.1109/TASL.2010.2103058

Articulatory information for noise robust speech recognition

Vikramjit Mitra^*
, Hosung Nam
, Carol Y. Espy-Wilson
, Elliot Saltzman
, Louis Goldstein

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

60 Citations (Scopus)

Abstract

Prior research has shown that articulatory information, if extracted properly from the speech signal, can improve the performance of automatic speech recognition systems. However, such information is not readily available in the signal. The challenge posed by the estimation of articulatory information from speech acoustics has led to a new line of research known as "acoustic-to- articulatory inversion" or "speech-inversion." While most of the research in this area has focused on estimating articulatory information more accurately, few have explored ways to apply this information in speech recognition tasks. In this paper, we first estimated articulatory information in the form of vocal tract constriction variables (abbreviated as TVs) from the Aurora-2 speech corpus using a neural network based speech-inversion model. Word recognition tasks were then performed for both noisy and clean speech using articulatory information in conjunction with traditional acoustic features. Our results indicate that incorporating TVs can significantly improve word recognition rates when used in conjunction with traditional acoustic features.

Original language	English
Article number	5677601
Pages (from-to)	1913-1924
Number of pages	12
Journal	IEEE Transactions on Audio, Speech and Language Processing
Volume	19
Issue number	7
DOIs	https://doi.org/10.1109/TASL.2010.2103058
Publication status	Published - 2011
Externally published	Yes

Bibliographical note

Funding Information:
Manuscript received March 15, 2010; revised July 06, 2010 and October 12, 2010; accepted December 08, 2010. Date of publication December 30, 2010; date of current version July 15, 2011. This work was supported in part by the National Science Foundation under Grants IIS0703859, IIS-0703048, and IIS0703782. V. Mitra and H. Nam contributed equally to this work. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Nestor Becerra Yoma.

Keywords

Articulatory phonology
articulatory speech recognition
artificial neural networks (ANNs)
noise-robust speech recognition
speech inversion
task dynamic model
vocal-tract variables

ASJC Scopus subject areas

Acoustics and Ultrasonics
Electrical and Electronic Engineering

Access to Document

10.1109/TASL.2010.2103058

Cite this

@article{27ad1632db204de3a1c057309b38cca5,

title = "Articulatory information for noise robust speech recognition",

abstract = "Prior research has shown that articulatory information, if extracted properly from the speech signal, can improve the performance of automatic speech recognition systems. However, such information is not readily available in the signal. The challenge posed by the estimation of articulatory information from speech acoustics has led to a new line of research known as {"}acoustic-to- articulatory inversion{"} or {"}speech-inversion.{"} While most of the research in this area has focused on estimating articulatory information more accurately, few have explored ways to apply this information in speech recognition tasks. In this paper, we first estimated articulatory information in the form of vocal tract constriction variables (abbreviated as TVs) from the Aurora-2 speech corpus using a neural network based speech-inversion model. Word recognition tasks were then performed for both noisy and clean speech using articulatory information in conjunction with traditional acoustic features. Our results indicate that incorporating TVs can significantly improve word recognition rates when used in conjunction with traditional acoustic features.",

keywords = "Articulatory phonology, articulatory speech recognition, artificial neural networks (ANNs), noise-robust speech recognition, speech inversion, task dynamic model, vocal-tract variables",

author = "Vikramjit Mitra and Hosung Nam and Espy-Wilson, \{Carol Y.\} and Elliot Saltzman and Louis Goldstein",

note = "Funding Information: Manuscript received March 15, 2010; revised July 06, 2010 and October 12, 2010; accepted December 08, 2010. Date of publication December 30, 2010; date of current version July 15, 2011. This work was supported in part by the National Science Foundation under Grants IIS0703859, IIS-0703048, and IIS0703782. V. Mitra and H. Nam contributed equally to this work. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Nestor Becerra Yoma.",

year = "2011",

doi = "10.1109/TASL.2010.2103058",

language = "English",

volume = "19",

pages = "1913--1924",

journal = "IEEE Transactions on Audio, Speech and Language Processing",

issn = "1558-7916",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "7",

}

TY - JOUR

T1 - Articulatory information for noise robust speech recognition

AU - Mitra, Vikramjit

AU - Nam, Hosung

AU - Espy-Wilson, Carol Y.

AU - Saltzman, Elliot

AU - Goldstein, Louis

N1 - Funding Information: Manuscript received March 15, 2010; revised July 06, 2010 and October 12, 2010; accepted December 08, 2010. Date of publication December 30, 2010; date of current version July 15, 2011. This work was supported in part by the National Science Foundation under Grants IIS0703859, IIS-0703048, and IIS0703782. V. Mitra and H. Nam contributed equally to this work. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Nestor Becerra Yoma.

PY - 2011

Y1 - 2011

N2 - Prior research has shown that articulatory information, if extracted properly from the speech signal, can improve the performance of automatic speech recognition systems. However, such information is not readily available in the signal. The challenge posed by the estimation of articulatory information from speech acoustics has led to a new line of research known as "acoustic-to- articulatory inversion" or "speech-inversion." While most of the research in this area has focused on estimating articulatory information more accurately, few have explored ways to apply this information in speech recognition tasks. In this paper, we first estimated articulatory information in the form of vocal tract constriction variables (abbreviated as TVs) from the Aurora-2 speech corpus using a neural network based speech-inversion model. Word recognition tasks were then performed for both noisy and clean speech using articulatory information in conjunction with traditional acoustic features. Our results indicate that incorporating TVs can significantly improve word recognition rates when used in conjunction with traditional acoustic features.

AB - Prior research has shown that articulatory information, if extracted properly from the speech signal, can improve the performance of automatic speech recognition systems. However, such information is not readily available in the signal. The challenge posed by the estimation of articulatory information from speech acoustics has led to a new line of research known as "acoustic-to- articulatory inversion" or "speech-inversion." While most of the research in this area has focused on estimating articulatory information more accurately, few have explored ways to apply this information in speech recognition tasks. In this paper, we first estimated articulatory information in the form of vocal tract constriction variables (abbreviated as TVs) from the Aurora-2 speech corpus using a neural network based speech-inversion model. Word recognition tasks were then performed for both noisy and clean speech using articulatory information in conjunction with traditional acoustic features. Our results indicate that incorporating TVs can significantly improve word recognition rates when used in conjunction with traditional acoustic features.

KW - Articulatory phonology

KW - articulatory speech recognition

KW - artificial neural networks (ANNs)

KW - noise-robust speech recognition

KW - speech inversion

KW - task dynamic model

KW - vocal-tract variables

UR - https://www.scopus.com/pages/publications/79960545035

U2 - 10.1109/TASL.2010.2103058

DO - 10.1109/TASL.2010.2103058

M3 - Article

AN - SCOPUS:79960545035

SN - 1558-7916

VL - 19

SP - 1913

EP - 1924

JO - IEEE Transactions on Audio, Speech and Language Processing

JF - IEEE Transactions on Audio, Speech and Language Processing

IS - 7

M1 - 5677601

ER -

Articulatory information for noise robust speech recognition

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this