BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text

Chanjun Park; Jaehyung Seo; Seolhwa Lee; Chanhee Lee; Hyeonseok Moon; Sugyeong Eo; Heuiseok Lim

BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text

Chanjun Park, Jaehyung Seo, Seolhwa Lee, Chanhee Lee, Hyeonseok Moon, Sugyeong Eo, Heuiseok Lim

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

With the growing popularity of smart speakers, such as Amazon Alexa, speech is becoming one of the most important modes of human-computer interaction. Automatic speech recognition (ASR) is arguably the most critical component of such systems, as errors in speech recognition propagate to the downstream components and drastically degrade the user experience. A simple and effective way to improve the speech recognition accuracy is to apply automatic post-processor to the recognition result. However, training a post-processor requires parallel corpora created by human annotators, which are expensive and not scalable. To alleviate this problem, we propose Back TranScription (BTS), a denoising-based method that can create such corpora without human labor. Using a raw corpus, BTS corrupts the text using Text-to-Speech (TTS) and Speech-to-Text (STT) systems. Then, a post-processing model can be trained to reconstruct the original text given the corrupted input. Quantitative and qualitative evaluations show that a post-processor trained using our approach is highly effective in fixing non-trivial speech recognition errors such as mishandling foreign words. We present the generated parallel corpus and post-processing platform to make our results publicly available.

Original language	English
Title of host publication	WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop
Editors	Toshiaki Nakazawa, Hideki Nakayama, Isao Goto, Hideya Mino, Chenchen Ding, Raj Dabre, Anoop Kunchukuttan, Shohei Higashiyama, Hiroshi Manabe, Win Pa Pa, Shantipriya Parida, Ondrej Bojar, Chenhui Chu, Akiko Eriguchi, Kaori Abe, Yusuke Oda, Katsuhito Sudoh, Sadao Kurohashi, Pushpak Bhattacharyya
Publisher	Association for Computational Linguistics (ACL)
Pages	106-116
Number of pages	11
ISBN (Electronic)	9781954085633
Publication status	Published - 2021
Event	8th Workshop on Asian Translation, WAT 2021 - Virtual, Bangkok, Thailand Duration: 2021 Aug 5 → 2021 Aug 6

Publication series

Name	WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop

Conference

Conference	8th Workshop on Asian Translation, WAT 2021
Country/Territory	Thailand
City	Virtual, Bangkok
Period	21/8/5 → 21/8/6

Bibliographical note

Funding Information:
This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-0-01405) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation), Institute for Information & communications Technology Planning & Evaluation (IITP), grant funded by the Korean government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques) and MSIT(Ministry of Science and ICT), Korea, under the ICT Creative Consilience program(IITP-2021-2020-0-01819) supervised by the IITP(Institute for Information & communications Technology Planning Evaluation).

Publisher Copyright:
© 2021 Association for Computational Linguistics.

ASJC Scopus subject areas

Language and Linguistics
Computational Theory and Mathematics
Computer Science Applications
Software

Cite this

Park, C., Seo, J., Lee, S., Lee, C., Moon, H., Eo, S., & Lim, H. (2021). BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text. In T. Nakazawa, H. Nakayama, I. Goto, H. Mino, C. Ding, R. Dabre, A. Kunchukuttan, S. Higashiyama, H. Manabe, W. P. Pa, S. Parida, O. Bojar, C. Chu, A. Eriguchi, K. Abe, Y. Oda, K. Sudoh, S. Kurohashi, & P. Bhattacharyya (Eds.), WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop (pp. 106-116). (WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop). Association for Computational Linguistics (ACL).

BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text. / Park, Chanjun; Seo, Jaehyung; Lee, Seolhwa et al.
WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop. ed. / Toshiaki Nakazawa; Hideki Nakayama; Isao Goto; Hideya Mino; Chenchen Ding; Raj Dabre; Anoop Kunchukuttan; Shohei Higashiyama; Hiroshi Manabe; Win Pa Pa; Shantipriya Parida; Ondrej Bojar; Chenhui Chu; Akiko Eriguchi; Kaori Abe; Yusuke Oda; Katsuhito Sudoh; Sadao Kurohashi; Pushpak Bhattacharyya. Association for Computational Linguistics (ACL), 2021. p. 106-116 (WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Park, C, Seo, J, Lee, S, Lee, C, Moon, H, Eo, S & Lim, H 2021, BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text. in T Nakazawa, H Nakayama, I Goto, H Mino, C Ding, R Dabre, A Kunchukuttan, S Higashiyama, H Manabe, WP Pa, S Parida, O Bojar, C Chu, A Eriguchi, K Abe, Y Oda, K Sudoh, S Kurohashi & P Bhattacharyya (eds), WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop. WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 106-116, 8th Workshop on Asian Translation, WAT 2021, Virtual, Bangkok, Thailand, 21/8/5.

Park C, Seo J, Lee S, Lee C, Moon H, Eo S et al. BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text. In Nakazawa T, Nakayama H, Goto I, Mino H, Ding C, Dabre R, Kunchukuttan A, Higashiyama S, Manabe H, Pa WP, Parida S, Bojar O, Chu C, Eriguchi A, Abe K, Oda Y, Sudoh K, Kurohashi S, Bhattacharyya P, editors, WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2021. p. 106-116. (WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop).

Park, Chanjun ; Seo, Jaehyung ; Lee, Seolhwa et al. / BTS : Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text. WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop. editor / Toshiaki Nakazawa ; Hideki Nakayama ; Isao Goto ; Hideya Mino ; Chenchen Ding ; Raj Dabre ; Anoop Kunchukuttan ; Shohei Higashiyama ; Hiroshi Manabe ; Win Pa Pa ; Shantipriya Parida ; Ondrej Bojar ; Chenhui Chu ; Akiko Eriguchi ; Kaori Abe ; Yusuke Oda ; Katsuhito Sudoh ; Sadao Kurohashi ; Pushpak Bhattacharyya. Association for Computational Linguistics (ACL), 2021. pp. 106-116 (WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop).

@inproceedings{c5b1fd5252bb425997b9237718d9cbb8,

title = "BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text",

abstract = "With the growing popularity of smart speakers, such as Amazon Alexa, speech is becoming one of the most important modes of human-computer interaction. Automatic speech recognition (ASR) is arguably the most critical component of such systems, as errors in speech recognition propagate to the downstream components and drastically degrade the user experience. A simple and effective way to improve the speech recognition accuracy is to apply automatic post-processor to the recognition result. However, training a post-processor requires parallel corpora created by human annotators, which are expensive and not scalable. To alleviate this problem, we propose Back TranScription (BTS), a denoising-based method that can create such corpora without human labor. Using a raw corpus, BTS corrupts the text using Text-to-Speech (TTS) and Speech-to-Text (STT) systems. Then, a post-processing model can be trained to reconstruct the original text given the corrupted input. Quantitative and qualitative evaluations show that a post-processor trained using our approach is highly effective in fixing non-trivial speech recognition errors such as mishandling foreign words. We present the generated parallel corpus and post-processing platform to make our results publicly available.",

author = "Chanjun Park and Jaehyung Seo and Seolhwa Lee and Chanhee Lee and Hyeonseok Moon and Sugyeong Eo and Heuiseok Lim",

note = "Funding Information: This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-0-01405) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation), Institute for Information & communications Technology Planning & Evaluation (IITP), grant funded by the Korean government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques) and MSIT(Ministry of Science and ICT), Korea, under the ICT Creative Consilience program(IITP-2021-2020-0-01819) supervised by the IITP(Institute for Information & communications Technology Planning Evaluation). Publisher Copyright: {\textcopyright} 2021 Association for Computational Linguistics.; 8th Workshop on Asian Translation, WAT 2021 ; Conference date: 05-08-2021 Through 06-08-2021",

year = "2021",

language = "English",

series = "WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "106--116",

editor = "Toshiaki Nakazawa and Hideki Nakayama and Isao Goto and Hideya Mino and Chenchen Ding and Raj Dabre and Anoop Kunchukuttan and Shohei Higashiyama and Hiroshi Manabe and Pa, {Win Pa} and Shantipriya Parida and Ondrej Bojar and Chenhui Chu and Akiko Eriguchi and Kaori Abe and Yusuke Oda and Katsuhito Sudoh and Sadao Kurohashi and Pushpak Bhattacharyya",

booktitle = "WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop",

}

TY - GEN

T1 - BTS

T2 - 8th Workshop on Asian Translation, WAT 2021

AU - Park, Chanjun

AU - Seo, Jaehyung

AU - Lee, Seolhwa

AU - Lee, Chanhee

AU - Moon, Hyeonseok

AU - Eo, Sugyeong

AU - Lim, Heuiseok

N1 - Funding Information: This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-0-01405) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation), Institute for Information & communications Technology Planning & Evaluation (IITP), grant funded by the Korean government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques) and MSIT(Ministry of Science and ICT), Korea, under the ICT Creative Consilience program(IITP-2021-2020-0-01819) supervised by the IITP(Institute for Information & communications Technology Planning Evaluation). Publisher Copyright: © 2021 Association for Computational Linguistics.

PY - 2021

Y1 - 2021

N2 - With the growing popularity of smart speakers, such as Amazon Alexa, speech is becoming one of the most important modes of human-computer interaction. Automatic speech recognition (ASR) is arguably the most critical component of such systems, as errors in speech recognition propagate to the downstream components and drastically degrade the user experience. A simple and effective way to improve the speech recognition accuracy is to apply automatic post-processor to the recognition result. However, training a post-processor requires parallel corpora created by human annotators, which are expensive and not scalable. To alleviate this problem, we propose Back TranScription (BTS), a denoising-based method that can create such corpora without human labor. Using a raw corpus, BTS corrupts the text using Text-to-Speech (TTS) and Speech-to-Text (STT) systems. Then, a post-processing model can be trained to reconstruct the original text given the corrupted input. Quantitative and qualitative evaluations show that a post-processor trained using our approach is highly effective in fixing non-trivial speech recognition errors such as mishandling foreign words. We present the generated parallel corpus and post-processing platform to make our results publicly available.

AB - With the growing popularity of smart speakers, such as Amazon Alexa, speech is becoming one of the most important modes of human-computer interaction. Automatic speech recognition (ASR) is arguably the most critical component of such systems, as errors in speech recognition propagate to the downstream components and drastically degrade the user experience. A simple and effective way to improve the speech recognition accuracy is to apply automatic post-processor to the recognition result. However, training a post-processor requires parallel corpora created by human annotators, which are expensive and not scalable. To alleviate this problem, we propose Back TranScription (BTS), a denoising-based method that can create such corpora without human labor. Using a raw corpus, BTS corrupts the text using Text-to-Speech (TTS) and Speech-to-Text (STT) systems. Then, a post-processing model can be trained to reconstruct the original text given the corrupted input. Quantitative and qualitative evaluations show that a post-processor trained using our approach is highly effective in fixing non-trivial speech recognition errors such as mishandling foreign words. We present the generated parallel corpus and post-processing platform to make our results publicly available.

UR - http://www.scopus.com/inward/record.url?scp=85115210038&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85115210038

T3 - WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop

SP - 106

EP - 116

BT - WAT 2021 - 8th Workshop on Asian Translation, Proceedings of the Workshop

A2 - Nakazawa, Toshiaki

A2 - Nakayama, Hideki

A2 - Goto, Isao

A2 - Mino, Hideya

A2 - Ding, Chenchen

A2 - Dabre, Raj

A2 - Kunchukuttan, Anoop

A2 - Higashiyama, Shohei

A2 - Manabe, Hiroshi

A2 - Pa, Win Pa

A2 - Parida, Shantipriya

A2 - Bojar, Ondrej

A2 - Chu, Chenhui

A2 - Eriguchi, Akiko

A2 - Abe, Kaori

A2 - Oda, Yusuke

A2 - Sudoh, Katsuhito

A2 - Kurohashi, Sadao

A2 - Bhattacharyya, Pushpak

PB - Association for Computational Linguistics (ACL)

Y2 - 5 August 2021 through 6 August 2021

ER -

BTS: Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this