Emotion recognition using convolutional neural network with selected statistical photoplethysmogram features

Min Seop Lee, Yun Kyu Lee, Myo Taeg Lim, Tae Koo Kang

Research output: Contribution to journalArticlepeer-review

44 Citations (Scopus)


Emotion recognition research has been conducted using various physiological signals. In this paper, we propose an efficient photoplethysmogram-based method that fuses the deep features extracted by two deep convolutional neural networks and the statistical features selected by Pearson's correlation technique. A photoplethysmogram (PPG) signal can be easily obtained through many devices, and the procedure for recording this signal is simpler than that for other physiological signals. The normal-to-normal (NN) interval values of heart rate variability (HRV) were utilized to extract the time domain features, and the normalized PPG signal was used to acquire the frequency domain features. Then, we selected features that correlated highly with an emotion through Pearson's correlation. These statistical features were fused with deep-learning features extracted from a convolutional neural network (CNN). The PPG signal and the NN interval were used as the inputs of the CNN to extract the features, and the total concatenated features were utilized to classify the valence and the arousal, which are the basic parameters of emotion. The Database for Emotion Analysis using Physiological signals (DEAP) was chosen for the experiment, and the results demonstrated that the proposed method achieved a noticeable performance with a short recognition interval.

Original languageEnglish
Article number3501
JournalApplied Sciences (Switzerland)
Issue number10
Publication statusPublished - 2020 May 1

Bibliographical note

Funding Information:
This research was supported by the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2019R1A2C1089742).

Publisher Copyright:
© 2020 by the authors.


  • Convolutional neural network
  • Emotion recognition
  • Feature fusion
  • PPG
  • Statistical feature

ASJC Scopus subject areas

  • General Materials Science
  • Instrumentation
  • General Engineering
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes


Dive into the research topics of 'Emotion recognition using convolutional neural network with selected statistical photoplethysmogram features'. Together they form a unique fingerprint.

Cite this