Autonomous System for EEG-Based Multiple Abnormal Mental States Classification Using Hybrid Deep Neural Networks under Flight Environment

Dae Hyeok Lee, Ji Hoon Jeong, Baek Woon Yu, Tae Eui Kam, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

Detection of the pilots' mental states is particularly critical because their abnormal mental states (AbSs) could cause catastrophic accidents. In this study, we presented the feasibility of classifying the various specific AbSs (namely, low fatigue, high fatigue, low workload, high workload, low distraction, and high distraction) by applying the deep learning method. To the best of our knowledge, this study is the first attempt to classify multiple AbSs of pilots. We proposed the hybrid deep neural networks with five convolutional blocks and two long short-term memory layers for decoding multiple AbSs. We designed the model to extract the informative features from electroencephalography signals. A total of ten pilots conducted the experiment in a simulated flight environment. Compared with five conventional models, our proposed model achieved the highest grand-average accuracy of 68.04(± 5.26)% which is at least 6.55% higher than other conventional models for classifying seven mental states across all subjects. Our proposed model could distinguish and classify low and high levels for each status category and give appropriate feedback to the subjects. In addition, we found nine indicators that showed the statistically significant differences between two mental states ( p < 0.05 ). Hence, we believe that it will contribute significantly to autonomous driving or autopilot advances based on artificial intelligence technology in the future.

Original languageEnglish
Pages (from-to)6426-6437
Number of pages12
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Volume53
Issue number10
DOIs
Publication statusPublished - 2023 Oct 1

Bibliographical note

Publisher Copyright:
© 2013 IEEE.

Keywords

  • Abnormal mental states (AbSs)
  • brain-computer interface (BCI)
  • electroencephalogram (EEG)
  • flight environment

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Autonomous System for EEG-Based Multiple Abnormal Mental States Classification Using Hybrid Deep Neural Networks under Flight Environment'. Together they form a unique fingerprint.

Cite this