Brain imaging studies of human speech are an active and intriguing research topic that is generating novel ways of communication through human brain signals. Efforts to generate voice from human neural activity have demonstrated the potential based on invasive measurements of speech, but have encountered difficulties in recreating data from imagined speech. Here, we propose NeuroTalk, which non-invasively converts brain signals from spoken and imagined speech to voice. The proposed framework is well-suited for decoding imagined speech, as it was trained on speech EEG data that was generalized to the domain of imagined speech. This means that the voice you hear when you imagine speaking is likely corresponding to the true voice of someone else, as the model has been specifically designed to adjust to this type of speech. Our findings suggest that speech synthesis of human EEG signals is a viable possibility, not just for spoken speech but also for imagined speech. This paper has extensively covered the contents of the paper, Lee et al. at AAAI 2023. Clearly, a high overlap with the above-mentioned contributions is inevitable and deliberate.
|Title of host publication||11th International Winter Conference on Brain-Computer Interface, BCI 2023|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Publication status||Published - 2023|
|Event||11th International Winter Conference on Brain-Computer Interface, BCI 2023 - Virtual, Online, Korea, Republic of|
Duration: 2023 Feb 20 → 2023 Feb 22
|Name||International Winter Conference on Brain-Computer Interface, BCI|
|Conference||11th International Winter Conference on Brain-Computer Interface, BCI 2023|
|Country/Territory||Korea, Republic of|
|Period||23/2/20 → 23/2/22|
Bibliographical noteFunding Information:
This work was partly supported by Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00451, Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning; No.2021-0-02068, Artificial Intelligence Innovation Hub).
© 2023 IEEE.
- computer interface
- generative model
- speech synthesis
ASJC Scopus subject areas
- Artificial Intelligence
- Human-Computer Interaction
- Signal Processing