ERAD-Fe: Emotion Recognition-Assisted Deep Learning Framework

Sun Hee Kim; Ngoc Anh Thi Nguyen; Hyung Jeong Yang; Seong Whan Lee

doi:10.1109/TIM.2021.3115195

ERAD-Fe: Emotion Recognition-Assisted Deep Learning Framework

Sun Hee Kim, Ngoc Anh Thi Nguyen, Hyung Jeong Yang, Seong Whan Lee

Department of Artificial Intelligence

Research output: Contribution to journal › Article › peer-review

9 Citations (Scopus)

Abstract

With recent advancements in artificial intelligence technologies and human-computer interaction, strategies to identify the inner emotional states of humans through physiological signals such as electroencephalography (EEG) have been actively investigated and applied in various fields. Thus, there is an increasing demand for emotion analysis and recognition via EEG signals in real time. In this article, we proposed a new framework, emotion recognition-assisted deep learning framework from eeg signal (eRAD-Fe), to achieve the best recognition result from EEG signals. eRAD-Fe integrates three aspects by exploiting sliding-window segmentation method to enlarge the size of the training dataset, configuring the energy threshold-based multiclass common spatial patterns to extract the prominent features, and improving emotional state recognition performance based on a long short-term memory model. With our proposed recognition-assisted framework, the emotional classification accuracies were 82%, 72%, and 81% on three publicly available EEG datasets, such as SEED, DEAP, and DREAMER, respectively.

Original language	English
Journal	IEEE Transactions on Instrumentation and Measurement
Volume	70
DOIs	https://doi.org/10.1109/TIM.2021.3115195
Publication status	Published - 2021

Bibliographical note

Funding Information:
This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education under Grant NRF-2021R1I1A1A01048455, in part by Vietnam National Foundation for Science and Technology Development (NAFOS-TED) under Grant 102.01-2020.27, in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant NRF-2020R1A4A1019191, and in part by the Institute for Information and Communications Technology Promotion (IITP) Grant funded by the Government of South Korea (Development of BCI-Based Brain and Cognitive Computing Technology for Recognizing User?s Intentions using Deep Learning under Grant 2017-0-00451; Artificial Intelligence Graduate School Program, Korea University, under Grant 2019-0- 00079).

Publisher Copyright:
© 1963-2012 IEEE.

Keywords

Deep learning
electroencephalogram
emotion recognition
energy threshold
feature extraction
long short-term memory
multiclass common spatial pattern (CSP)

ASJC Scopus subject areas

Instrumentation
Electrical and Electronic Engineering

Access to Document

10.1109/TIM.2021.3115195

Cite this

@article{221609adb83a43d7b12858f3838ea6ab,

title = "ERAD-Fe: Emotion Recognition-Assisted Deep Learning Framework",

abstract = "With recent advancements in artificial intelligence technologies and human-computer interaction, strategies to identify the inner emotional states of humans through physiological signals such as electroencephalography (EEG) have been actively investigated and applied in various fields. Thus, there is an increasing demand for emotion analysis and recognition via EEG signals in real time. In this article, we proposed a new framework, emotion recognition-assisted deep learning framework from eeg signal (eRAD-Fe), to achieve the best recognition result from EEG signals. eRAD-Fe integrates three aspects by exploiting sliding-window segmentation method to enlarge the size of the training dataset, configuring the energy threshold-based multiclass common spatial patterns to extract the prominent features, and improving emotional state recognition performance based on a long short-term memory model. With our proposed recognition-assisted framework, the emotional classification accuracies were 82%, 72%, and 81% on three publicly available EEG datasets, such as SEED, DEAP, and DREAMER, respectively.",

keywords = "Deep learning, electroencephalogram, emotion recognition, energy threshold, feature extraction, long short-term memory, multiclass common spatial pattern (CSP)",

author = "Kim, {Sun Hee} and Nguyen, {Ngoc Anh Thi} and Yang, {Hyung Jeong} and Lee, {Seong Whan}",

note = "Funding Information: This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education under Grant NRF-2021R1I1A1A01048455, in part by Vietnam National Foundation for Science and Technology Development (NAFOS-TED) under Grant 102.01-2020.27, in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant NRF-2020R1A4A1019191, and in part by the Institute for Information and Communications Technology Promotion (IITP) Grant funded by the Government of South Korea (Development of BCI-Based Brain and Cognitive Computing Technology for Recognizing User?s Intentions using Deep Learning under Grant 2017-0-00451; Artificial Intelligence Graduate School Program, Korea University, under Grant 2019-0- 00079). Publisher Copyright: {\textcopyright} 1963-2012 IEEE.",

year = "2021",

doi = "10.1109/TIM.2021.3115195",

language = "English",

volume = "70",

journal = "IEEE Transactions on Instrumentation and Measurement",

issn = "0018-9456",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - ERAD-Fe

T2 - Emotion Recognition-Assisted Deep Learning Framework

AU - Kim, Sun Hee

AU - Nguyen, Ngoc Anh Thi

AU - Yang, Hyung Jeong

AU - Lee, Seong Whan

N1 - Funding Information: This work was supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education under Grant NRF-2021R1I1A1A01048455, in part by Vietnam National Foundation for Science and Technology Development (NAFOS-TED) under Grant 102.01-2020.27, in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant NRF-2020R1A4A1019191, and in part by the Institute for Information and Communications Technology Promotion (IITP) Grant funded by the Government of South Korea (Development of BCI-Based Brain and Cognitive Computing Technology for Recognizing User?s Intentions using Deep Learning under Grant 2017-0-00451; Artificial Intelligence Graduate School Program, Korea University, under Grant 2019-0- 00079). Publisher Copyright: © 1963-2012 IEEE.

PY - 2021

Y1 - 2021

N2 - With recent advancements in artificial intelligence technologies and human-computer interaction, strategies to identify the inner emotional states of humans through physiological signals such as electroencephalography (EEG) have been actively investigated and applied in various fields. Thus, there is an increasing demand for emotion analysis and recognition via EEG signals in real time. In this article, we proposed a new framework, emotion recognition-assisted deep learning framework from eeg signal (eRAD-Fe), to achieve the best recognition result from EEG signals. eRAD-Fe integrates three aspects by exploiting sliding-window segmentation method to enlarge the size of the training dataset, configuring the energy threshold-based multiclass common spatial patterns to extract the prominent features, and improving emotional state recognition performance based on a long short-term memory model. With our proposed recognition-assisted framework, the emotional classification accuracies were 82%, 72%, and 81% on three publicly available EEG datasets, such as SEED, DEAP, and DREAMER, respectively.

AB - With recent advancements in artificial intelligence technologies and human-computer interaction, strategies to identify the inner emotional states of humans through physiological signals such as electroencephalography (EEG) have been actively investigated and applied in various fields. Thus, there is an increasing demand for emotion analysis and recognition via EEG signals in real time. In this article, we proposed a new framework, emotion recognition-assisted deep learning framework from eeg signal (eRAD-Fe), to achieve the best recognition result from EEG signals. eRAD-Fe integrates three aspects by exploiting sliding-window segmentation method to enlarge the size of the training dataset, configuring the energy threshold-based multiclass common spatial patterns to extract the prominent features, and improving emotional state recognition performance based on a long short-term memory model. With our proposed recognition-assisted framework, the emotional classification accuracies were 82%, 72%, and 81% on three publicly available EEG datasets, such as SEED, DEAP, and DREAMER, respectively.

KW - Deep learning

KW - electroencephalogram

KW - emotion recognition

KW - energy threshold

KW - feature extraction

KW - long short-term memory

KW - multiclass common spatial pattern (CSP)

UR - http://www.scopus.com/inward/record.url?scp=85115730479&partnerID=8YFLogxK

U2 - 10.1109/TIM.2021.3115195

DO - 10.1109/TIM.2021.3115195

M3 - Article

AN - SCOPUS:85115730479

SN - 0018-9456

VL - 70

JO - IEEE Transactions on Instrumentation and Measurement

JF - IEEE Transactions on Instrumentation and Measurement

ER -

ERAD-Fe: Emotion Recognition-Assisted Deep Learning Framework

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this