Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment

Byoung Hee Kwon; Ji Hoon Jeong; Jeong Hyun Cho; Seong Whan Lee

doi:10.1109/SMC42975.2020.9282814

Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment

Byoung Hee Kwon, Ji Hoon Jeong, Jeong Hyun Cho, Seong Whan Lee

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

10 Citations (Scopus)

Abstract

In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we commonly used in daily life, such as picking up a phone, opening a door, eating food, and pouring water. Nine subjects participated in our experiment. We presented statistical evidence that visual motion imagery has a high correlation from the prefrontal and occipital lobes. In addition, we selected the most appropriate electroencephalography channels using a functional connectivity approach for visual motion imagery decoding and proposed a convolutional neural network architecture for classification. As a result, the averaged classification performance of the proposed architecture for 4 classes from 16 channels was 67.50 (±1.52)% across all subjects. This result is encouraging, and it shows the possibility of developing a BCI-based device control system for practical applications such as neuroprosthesis and a robotic arm.

Original language	English
Title of host publication	2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2966-2971
Number of pages	6
ISBN (Electronic)	9781728185262
DOIs	https://doi.org/10.1109/SMC42975.2020.9282814
Publication status	Published - 2020 Oct 11
Event	2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020 - Toronto, Canada Duration: 2020 Oct 11 → 2020 Oct 14

Publication series

Name	Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
Volume	2020-October
ISSN (Print)	1062-922X

Conference

Conference	2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020
Country/Territory	Canada
City	Toronto
Period	20/10/11 → 20/10/14

Bibliographical note

Funding Information:
This work was partly supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00432, Development of NonInvasive Integrated BCI SW Platform to Control Home Appliances and External Devices by User’s Thought via AR/VR Interface; No. 2017-0-00451, Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning; No. 2019-0-00079, Artificial Intelligence Graduate School Program (Korea University))

Publisher Copyright:
© 2020 IEEE.

Keywords

brain-computer interface
electroencephalography
functional connectivity
visual motion imagery

ASJC Scopus subject areas

Electrical and Electronic Engineering
Control and Systems Engineering
Human-Computer Interaction

Access to Document

10.1109/SMC42975.2020.9282814

Cite this

Kwon, B. H., Jeong, J. H., Cho, J. H., & Lee, S. W. (2020). Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment. In 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020 (pp. 2966-2971). Article 9282814 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics; Vol. 2020-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC42975.2020.9282814

Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment. / Kwon, Byoung Hee; Jeong, Ji Hoon; Cho, Jeong Hyun et al.
2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020. Institute of Electrical and Electronics Engineers Inc., 2020. p. 2966-2971 9282814 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics; Vol. 2020-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kwon, BH, Jeong, JH, Cho, JH & Lee, SW 2020, Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment. in 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020., 9282814, Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, vol. 2020-October, Institute of Electrical and Electronics Engineers Inc., pp. 2966-2971, 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020, Toronto, Canada, 20/10/11. https://doi.org/10.1109/SMC42975.2020.9282814

Kwon BH, Jeong JH, Cho JH, Lee SW. Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment. In 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020. Institute of Electrical and Electronics Engineers Inc. 2020. p. 2966-2971. 9282814. (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). doi: 10.1109/SMC42975.2020.9282814

Kwon, Byoung Hee ; Jeong, Ji Hoon ; Cho, Jeong Hyun et al. / Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment. 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 2966-2971 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).

@inproceedings{3968ed64bc9f4b89a588a7556ea2b463,

title = "Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment",

abstract = "In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we commonly used in daily life, such as picking up a phone, opening a door, eating food, and pouring water. Nine subjects participated in our experiment. We presented statistical evidence that visual motion imagery has a high correlation from the prefrontal and occipital lobes. In addition, we selected the most appropriate electroencephalography channels using a functional connectivity approach for visual motion imagery decoding and proposed a convolutional neural network architecture for classification. As a result, the averaged classification performance of the proposed architecture for 4 classes from 16 channels was 67.50 (±1.52)% across all subjects. This result is encouraging, and it shows the possibility of developing a BCI-based device control system for practical applications such as neuroprosthesis and a robotic arm.",

keywords = "brain-computer interface, electroencephalography, functional connectivity, visual motion imagery",

author = "Kwon, {Byoung Hee} and Jeong, {Ji Hoon} and Cho, {Jeong Hyun} and Lee, {Seong Whan}",

note = "Funding Information: This work was partly supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00432, Development of NonInvasive Integrated BCI SW Platform to Control Home Appliances and External Devices by User{\textquoteright}s Thought via AR/VR Interface; No. 2017-0-00451, Development of BCI based Brain and Cognitive Computing Technology for Recognizing User{\textquoteright}s Intentions using Deep Learning; No. 2019-0-00079, Artificial Intelligence Graduate School Program (Korea University)) Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020 ; Conference date: 11-10-2020 Through 14-10-2020",

year = "2020",

month = oct,

day = "11",

doi = "10.1109/SMC42975.2020.9282814",

language = "English",

series = "Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2966--2971",

booktitle = "2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020",

}

TY - GEN

T1 - Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment

AU - Kwon, Byoung Hee

AU - Jeong, Ji Hoon

AU - Cho, Jeong Hyun

AU - Lee, Seong Whan

N1 - Funding Information: This work was partly supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00432, Development of NonInvasive Integrated BCI SW Platform to Control Home Appliances and External Devices by User’s Thought via AR/VR Interface; No. 2017-0-00451, Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning; No. 2019-0-00079, Artificial Intelligence Graduate School Program (Korea University)) Publisher Copyright: © 2020 IEEE.

PY - 2020/10/11

Y1 - 2020/10/11

N2 - In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we commonly used in daily life, such as picking up a phone, opening a door, eating food, and pouring water. Nine subjects participated in our experiment. We presented statistical evidence that visual motion imagery has a high correlation from the prefrontal and occipital lobes. In addition, we selected the most appropriate electroencephalography channels using a functional connectivity approach for visual motion imagery decoding and proposed a convolutional neural network architecture for classification. As a result, the averaged classification performance of the proposed architecture for 4 classes from 16 channels was 67.50 (±1.52)% across all subjects. This result is encouraging, and it shows the possibility of developing a BCI-based device control system for practical applications such as neuroprosthesis and a robotic arm.

AB - In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we commonly used in daily life, such as picking up a phone, opening a door, eating food, and pouring water. Nine subjects participated in our experiment. We presented statistical evidence that visual motion imagery has a high correlation from the prefrontal and occipital lobes. In addition, we selected the most appropriate electroencephalography channels using a functional connectivity approach for visual motion imagery decoding and proposed a convolutional neural network architecture for classification. As a result, the averaged classification performance of the proposed architecture for 4 classes from 16 channels was 67.50 (±1.52)% across all subjects. This result is encouraging, and it shows the possibility of developing a BCI-based device control system for practical applications such as neuroprosthesis and a robotic arm.

KW - brain-computer interface

KW - electroencephalography

KW - functional connectivity

KW - visual motion imagery

UR - http://www.scopus.com/inward/record.url?scp=85098844910&partnerID=8YFLogxK

U2 - 10.1109/SMC42975.2020.9282814

DO - 10.1109/SMC42975.2020.9282814

M3 - Conference contribution

AN - SCOPUS:85098844910

T3 - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

SP - 2966

EP - 2971

BT - 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020

Y2 - 11 October 2020 through 14 October 2020

ER -

Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this