Decoding Imagined Speech Based on Deep Metric Learning for Intuitive BCI Communication

Dong Yeon Lee, Minji Lee, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

17 Citations (Scopus)


Imagined speech is a highly promising paradigm due to its intuitive application and multiclass scalability in the field of brain-computer interfaces. However, optimal feature extraction and classifiers have not yet been established. Furthermore, retraining still requires a large number of trials when new classes are added. The aim of this study is (i) to increase the classification performance for imagined speech and (ii) to apply a new class using a pretrained classifier with a small number of trials. We propose a novel framework based on deep metric learning that learns the distance by comparing the similarity between samples. We also applied the instantaneous frequency and spectral entropy used for speech signals to electroencephalography signals during imagined speech. The method was evaluated on two public datasets (6-class Coretto DB and 5-class BCI Competition DB). We achieved a 6-class accuracy of 45.00 ± 3.13% and a 5-class accuracy of 48.10 ± 3.68% using the proposed method, which significantly outperformed state-of-the-art methods. Additionally, we verified that the new class could be detected through incremental learning with a small number of trials. As a result, the average accuracy is 44.50 ± 0.26% for Coretto DB and 47.12 ± 0.27% for BCI Competition DB, which shows similar accuracy to baseline accuracy without incremental learning. Our results have shown that the accuracy can be greatly improved even with a small number of trials by selecting appropriate features from imagined speech. The proposed framework could be directly used to help construct an extensible intuitive communication system based on brain-computer interfaces.

Original languageEnglish
Article number9481777
Pages (from-to)1363-1374
Number of pages12
JournalIEEE Transactions on Neural Systems and Rehabilitation Engineering
Publication statusPublished - 2021

Bibliographical note

Funding Information:
Manuscript received January 13, 2021; revised April 24, 2021 and June 15, 2021; accepted July 9, 2021. Date of publication July 13, 2021; date of current version July 20, 2021. This work was supported in part by the Institute for Information and Communications Technology Promotion (IITP) Grant through the Korean Government (Development of BCI-based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning) under Grant 2017-0-00451 and in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) Grant through the Korean Government (MSIT) (Artificial Intelligence Graduate School Program (Korea University)) under Grant 2019-0-00079. (Dong-Yeon Lee and Minji Lee contributed equally to this work.) (Corresponding author: Seong-Whan Lee.) Dong-Yeon Lee and Minji Lee are with the Department of Brain and Cognitive Engineering, Korea University, Seongbuk-gu, Seoul 02841, Republic of Korea (e-mail:; minjilee@

Publisher Copyright:
© 2001-2011 IEEE.


  • Imagined speech
  • brain-computer interface
  • deep metric learning
  • instantaneous frequency
  • spectral entropy

ASJC Scopus subject areas

  • Internal Medicine
  • General Neuroscience
  • Biomedical Engineering
  • Rehabilitation


Dive into the research topics of 'Decoding Imagined Speech Based on Deep Metric Learning for Intuitive BCI Communication'. Together they form a unique fingerprint.

Cite this