Reinforce-Aligner: Reinforcement alignment search for robust end-to-end text-to-speech

Hyunseung Chung, Sang Hoon Lee, Seong Whan Lee

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    12 Citations (Scopus)

    Abstract

    Text-to-speech (TTS) synthesis is the process of producing synthesized speech from text or phoneme input. Traditional TTS models contain multiple processing steps and require external aligners, which provide attention alignments of phoneme-to-frame sequences. As the complexity increases and efficiency decreases with every additional step, there is expanding demand in modern synthesis pipelines for end-to-end TTS with efficient internal aligners. In this work, we propose an end-to-end text-to-waveform network with a novel reinforcement learning based duration search method. Our proposed generator is feed-forward and the aligner trains the agent to make optimal duration predictions by receiving active feedback from actions taken to maximize cumulative reward. We demonstrate accurate alignments of phoneme-to-frame sequence generated from trained agents enhance fidelity and naturalness of synthesized audio. Experimental results also show the superiority of our proposed model compared to other state-of-the-art TTS models with internal and external aligners.

    Original languageEnglish
    Title of host publication22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021
    PublisherInternational Speech Communication Association
    Pages3556-3560
    Number of pages5
    ISBN (Electronic)9781713836902
    DOIs
    Publication statusPublished - 2021
    Event22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021 - Brno, Czech Republic
    Duration: 2021 Aug 302021 Sept 3

    Publication series

    NameProceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
    Volume5
    ISSN (Print)2308-457X
    ISSN (Electronic)1990-9772

    Conference

    Conference22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021
    Country/TerritoryCzech Republic
    CityBrno
    Period21/8/3021/9/3

    Bibliographical note

    Funding Information:
    This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00079, Department of Artificial Intelligence, Korea University), and the Netmarble AI Center.

    Publisher Copyright:
    © 2021 ISCA

    Keywords

    • Reinforcement learning
    • Text to speech

    ASJC Scopus subject areas

    • Language and Linguistics
    • Human-Computer Interaction
    • Signal Processing
    • Software
    • Modelling and Simulation

    Fingerprint

    Dive into the research topics of 'Reinforce-Aligner: Reinforcement alignment search for robust end-to-end text-to-speech'. Together they form a unique fingerprint.

    Cite this