Retrieving tract variables from acoustics: A comparison of different machine learning strategies

Vikramjit Mitra; Hosung Nam; Carol Y. Espy-Wilson; Elliot Saltzman; Louis Goldstein

doi:10.1109/JSTSP.2010.2076013

Retrieving tract variables from acoustics: A comparison of different machine learning strategies

Vikramjit Mitra, Hosung Nam, Carol Y. Espy-Wilson, Elliot Saltzman, Louis Goldstein

Research output: Contribution to journal › Article › peer-review

47 Citations (Scopus)

Abstract

Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

Original language	English
Article number	5570879
Pages (from-to)	1027-1045
Number of pages	19
Journal	IEEE Journal on Selected Topics in Signal Processing
Volume	4
Issue number	6
DOIs	https://doi.org/10.1109/JSTSP.2010.2076013
Publication status	Published - 2010 Dec
Externally published	Yes

Bibliographical note

Funding Information:
Manuscript received December 15, 2009; accepted February 18, 2010. Date of publication September 13, 2010; date of current version November 17, 2010. This work was supported by National Science Foundation (NSF) under Grants IIS-0703859, IIS-0703048, IIS-0703782, and NIH-NIDCD grant DC-02717. V. Mitra and H. Nam contributed equally to this work. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Li Deng.

Keywords

Articulatory phonology
articulatory speech recognition (ASR)
artificial neural networks (ANNs)
coarticulation
distal supervised learning
mixture density networks
speech inversion
task dynamic and applications model
vocal-tract variables

ASJC Scopus subject areas

Signal Processing
Electrical and Electronic Engineering

Access to Document

10.1109/JSTSP.2010.2076013

Cite this

@article{8cce4264f8fd404889db0d7eb11ed983,

title = "Retrieving tract variables from acoustics: A comparison of different machine learning strategies",

abstract = "Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.",

keywords = "Articulatory phonology, articulatory speech recognition (ASR), artificial neural networks (ANNs), coarticulation, distal supervised learning, mixture density networks, speech inversion, task dynamic and applications model, vocal-tract variables",

author = "Vikramjit Mitra and Hosung Nam and Espy-Wilson, {Carol Y.} and Elliot Saltzman and Louis Goldstein",

note = "Funding Information: Manuscript received December 15, 2009; accepted February 18, 2010. Date of publication September 13, 2010; date of current version November 17, 2010. This work was supported by National Science Foundation (NSF) under Grants IIS-0703859, IIS-0703048, IIS-0703782, and NIH-NIDCD grant DC-02717. V. Mitra and H. Nam contributed equally to this work. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Li Deng.",

year = "2010",

month = dec,

doi = "10.1109/JSTSP.2010.2076013",

language = "English",

volume = "4",

pages = "1027--1045",

journal = "IEEE Journal on Selected Topics in Signal Processing",

issn = "1932-4553",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "6",

}

TY - JOUR

T1 - Retrieving tract variables from acoustics

T2 - A comparison of different machine learning strategies

AU - Mitra, Vikramjit

AU - Nam, Hosung

AU - Espy-Wilson, Carol Y.

AU - Saltzman, Elliot

AU - Goldstein, Louis

N1 - Funding Information: Manuscript received December 15, 2009; accepted February 18, 2010. Date of publication September 13, 2010; date of current version November 17, 2010. This work was supported by National Science Foundation (NSF) under Grants IIS-0703859, IIS-0703048, IIS-0703782, and NIH-NIDCD grant DC-02717. V. Mitra and H. Nam contributed equally to this work. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Li Deng.

PY - 2010/12

Y1 - 2010/12

N2 - Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

AB - Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

KW - Articulatory phonology

KW - articulatory speech recognition (ASR)

KW - artificial neural networks (ANNs)

KW - coarticulation

KW - distal supervised learning

KW - mixture density networks

KW - speech inversion

KW - task dynamic and applications model

KW - vocal-tract variables

UR - http://www.scopus.com/inward/record.url?scp=78649390043&partnerID=8YFLogxK

U2 - 10.1109/JSTSP.2010.2076013

DO - 10.1109/JSTSP.2010.2076013

M3 - Article

AN - SCOPUS:78649390043

SN - 1932-4553

VL - 4

SP - 1027

EP - 1045

JO - IEEE Journal on Selected Topics in Signal Processing

JF - IEEE Journal on Selected Topics in Signal Processing

IS - 6

M1 - 5570879

ER -

Retrieving tract variables from acoustics: A comparison of different machine learning strategies

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this