Sleep-CMKD: Self-Attention CNN/Transformer Cross-Model Knowledge Distillation for Automatic Sleep Staging

Hyounggyu Kim, Moogyeong Kim, Wonzoo Chung

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we propose single-channel cross-model knowledge distillation(CMKD) method between convolutional neural networks-based and transformer-based models for automatic sleep staging. In sleep staging, few works proposed to distill knowledge from additional sleep dataset or multi-channel polysomnogram requiring manual scoring effort of human experts and limiting the home application senarios. Cross-model knowledge distillation avoids these additional efforts and limitations by distilling inductive biases between models with different structures. Experiments on Sleep-EDFX-78 dataset confirm that the proposed method improves sleep stage classification accuracy of transformer-based model by 1.7%.

Original languageEnglish
Title of host publication11th International Winter Conference on Brain-Computer Interface, BCI 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665464444
DOIs
Publication statusPublished - 2023
Event11th International Winter Conference on Brain-Computer Interface, BCI 2023 - Virtual, Online, Korea, Republic of
Duration: 2023 Feb 202023 Feb 22

Publication series

NameInternational Winter Conference on Brain-Computer Interface, BCI
Volume2023-February
ISSN (Print)2572-7672

Conference

Conference11th International Winter Conference on Brain-Computer Interface, BCI 2023
Country/TerritoryKorea, Republic of
CityVirtual, Online
Period23/2/2023/2/22

Bibliographical note

Funding Information:
This work was partly supported by Institute of Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (No. 2019-0-00079, Artificial Intelligence Graduate School Program(Korea University)), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00432, Development Of Non-invasive Integrated BCI SW Platform To Control Home Appliance And External Devices By User’s Thought Via AR/VR Interface), Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00451, Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2021-0-02068, Artificial Intelligence Innovation Hub), and the BK21 four program through the National Research Foundation (NRF) funded by the Ministry of Education of Korea.

Publisher Copyright:
© 2023 IEEE.

Keywords

  • Automatic Sleep Staging
  • Electroencephalogram (EEG)
  • Knowledge Distillation
  • Transformer

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Signal Processing

Fingerprint

Dive into the research topics of 'Sleep-CMKD: Self-Attention CNN/Transformer Cross-Model Knowledge Distillation for Automatic Sleep Staging'. Together they form a unique fingerprint.

Cite this