Spatio-Spectral Feature Representation for Motor Imagery Classification Using Convolutional Neural Networks

Ji Seon Bang, Min Ho Lee, Siamac Fazli, Cuntai Guan, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

45 Citations (Scopus)


Convolutional neural networks (CNNs) have recently been applied to electroencephalogram (EEG)-based brain-computer interfaces (BCIs). EEG is a noninvasive neuroimaging technique, which can be used to decode user intentions. Because the feature space of EEG data is highly dimensional and signal patterns are specific to the subject, appropriate methods for feature representation are required to enhance the decoding accuracy of the CNN model. Furthermore, neural changes exhibit high variability between sessions, subjects within a single session, and trials within a single subject, resulting in major issues during the modeling stage. In addition, there are many subject-dependent factors, such as frequency ranges, time intervals, and spatial locations at which the signal occurs, which prevent the derivation of a robust model that can achieve the parameterization of these factors for a wide range of subjects. However, previous studies did not attempt to preserve the multivariate structure and dependencies of the feature space. In this study, we propose a method to generate a spatiospectral feature representation that can preserve the multivariate information of EEG data. Specifically, 3-D feature maps were constructed by combining subject-optimized and subject-independent spectral filters and by stacking the filtered data into tensors. In addition, a layer-wise decomposition model was implemented using our 3-D-CNN framework to secure reliable classification results on a single-trial basis. The average accuracies of the proposed model were 87.15% (±7.31), 75.85% (±12.80), and 70.37% (±17.09) for the BCI competition data sets IV_2a, IV_2b, and OpenBMI data, respectively. These results are better than those obtained by state-of-the-art techniques, and the decomposition model obtained the relevance scores for neurophysiologically plausible electrode channels and frequency domains, confirming the validity of the proposed approach.

Original languageEnglish
Pages (from-to)3038-3049
Number of pages12
JournalIEEE Transactions on Neural Networks and Learning Systems
Issue number7
Publication statusPublished - 2022 Jul 1

Bibliographical note

Publisher Copyright:
© 2012 IEEE.


  • Brain-computer interface (BCI)
  • convolutional neural network (CNN)
  • electroencephalography (EEG)
  • explainable artificial intelligence (XAI)
  • motor imagery (MI)

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Spatio-Spectral Feature Representation for Motor Imagery Classification Using Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this