Continuous EEG Decoding of Pilots' Mental States Using Multiple Feature Block-Based Convolutional Neural Network

Dae Hyeok Lee, Ji Hoon Jeong, Kiduk Kim, Baek Woon Yu, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

50 Citations (Scopus)


Non-invasive brain-computer interface (BCI) has been developed for recognizing and classifying human mental states with high performances. Specifically, classifying pilots' mental states accurately is a critical issue because their cognitive states, which are induced by mental fatigue, workload, and distraction, may be fundamental in catastrophic accidents. In this study, we present an electroencephalogram (EEG) classification of four mental states (fatigue, workload, distraction, and the normal state) from EEG signals in both offline and pseudo-online analyses. To the best of our knowledge, this study is the first attempt to classify pilots' mental states using only EEG signals during continuous decoding. We recorded EEG signals from seven pilots under various simulated flight conditions. We proposed a multiple feature block-based convolutional neural network (MFB-CNN) with temporal-spatio EEG filters to recognize the pilot's current mental states. We validated the proposed method for two analyses across all subjects. In the offline analysis, we confirmed the classification accuracy of 0.75 (±0.04). Also, in the pseudo-online analysis, we obtained the detection accuracy of 0.72 (±0.20), 0.72 (±0.27), and 0.61 (±0.18) for fatigue, workload, and distraction, respectively. Hence, we demonstrate the feasibility of classifying various types of mental states for implementation in real-world environments.

Original languageEnglish
Article number9133061
Pages (from-to)121929-121941
Number of pages13
JournalIEEE Access
Publication statusPublished - 2020

Bibliographical note

Funding Information:
This work was supported in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) grant funded by the Korean Government under Grant 2017-0-00451 and Grant 2019-0-00079 and in part by the Defense Acquisition Program Administration (DAPA) and Agency for Defense Development (ADD) of Korea (A Study on Human–Computer Interaction Technology for the Pilot Status Recognition) under Grant 06-201-305-001.

Publisher Copyright:
© 2013 IEEE.


  • Brain-computer interface (BCI)
  • deep convolutional neural network
  • electroencephalogram (EEG)
  • mental states

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Continuous EEG Decoding of Pilots' Mental States Using Multiple Feature Block-Based Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this