Speech Synthesis from Brain Signals Based on Generative Model

Young Eun Lee, Sang Ho Kim, Seo Hyun Lee, Jung Sun Lee, Soowon Kim, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Brain imaging studies of human speech are an active and intriguing research topic that is generating novel ways of communication through human brain signals. Efforts to generate voice from human neural activity have demonstrated the potential based on invasive measurements of speech, but have encountered difficulties in recreating data from imagined speech. Here, we propose NeuroTalk, which non-invasively converts brain signals from spoken and imagined speech to voice. The proposed framework is well-suited for decoding imagined speech, as it was trained on speech EEG data that was generalized to the domain of imagined speech. This means that the voice you hear when you imagine speaking is likely corresponding to the true voice of someone else, as the model has been specifically designed to adjust to this type of speech. Our findings suggest that speech synthesis of human EEG signals is a viable possibility, not just for spoken speech but also for imagined speech. This paper has extensively covered the contents of the paper, Lee et al. at AAAI 2023. Clearly, a high overlap with the above-mentioned contributions is inevitable and deliberate.

Original languageEnglish
Title of host publication11th International Winter Conference on Brain-Computer Interface, BCI 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665464444
Publication statusPublished - 2023
Event11th International Winter Conference on Brain-Computer Interface, BCI 2023 - Virtual, Online, Korea, Republic of
Duration: 2023 Feb 202023 Feb 22

Publication series

NameInternational Winter Conference on Brain-Computer Interface, BCI
ISSN (Print)2572-7672


Conference11th International Winter Conference on Brain-Computer Interface, BCI 2023
Country/TerritoryKorea, Republic of
CityVirtual, Online

Bibliographical note

Funding Information:
This work was partly supported by Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00451, Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning; No.2021-0-02068, Artificial Intelligence Innovation Hub).

Publisher Copyright:
© 2023 IEEE.


  • computer interface
  • generative model
  • speech synthesis

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Signal Processing


Dive into the research topics of 'Speech Synthesis from Brain Signals Based on Generative Model'. Together they form a unique fingerprint.

Cite this