Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces

Dong Ok Won; Han Jeong Hwang; Dong Min Kim; Klaus Robert Müller; Seong Whan Lee

doi:10.1109/TNSRE.2017.2736600

Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces

Dong Ok Won, Han Jeong Hwang, Dong Min Kim, Klaus Robert Müller, Seong Whan Lee

Research output: Contribution to journal › Article › peer-review

98 Citations (Scopus)

Abstract

Most event-related potential (ERP)-based brain-computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (i.e., 360.4-379.6 ms; 5.5867- 5.7662μ V) than the NM-RSVP (i.e., 480.4 ms; 4.7426μ V). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (i.e., 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.

Original language	English
Article number	8008839
Pages (from-to)	334-343
Number of pages	10
Journal	IEEE Transactions on Neural Systems and Rehabilitation Engineering
Volume	26
Issue number	2
DOIs	https://doi.org/10.1109/TNSRE.2017.2736600
Publication status	Published - 2018 Feb

Bibliographical note

Funding Information:
Manuscript received May 4, 2016; revised November 7, 2016, March 14, 2017, and July 23, 2017; accepted July 24, 2017. Date of publication August 11, 2017; date of current version February 9, 2018. This work was supported in part by the Institute for Information and Communications Technology Promotion (IITP) through the Korea Government (MSIT) under Grant 2017-0-00451, in part by the Development of BCI-based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning) and the MSIT, South Korea, under the SW Starlab support program supervised by the IITP under Grant IITP-2015-1107. (Corresponding author: Seong-Whan Lee.) D.-O. Won, D.-M. Kim, and S.-W. Lee are with the Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea (e-mail: [email protected]; [email protected]; [email protected]).

Publisher Copyright:
© 2001-2011 IEEE.

Keywords

Brain-computer interface (BCI)
event-related potential (ERP)
gaze-independent
rapid serial visual presentation (RSVP)

ASJC Scopus subject areas

Internal Medicine
General Neuroscience
Biomedical Engineering

Access to Document

10.1109/TNSRE.2017.2736600

Cite this

@article{3c1b2d17d460418c8f658d58f4c209f9,

title = "Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces",

abstract = "Most event-related potential (ERP)-based brain-computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (i.e., 360.4-379.6 ms; 5.5867- 5.7662μ V) than the NM-RSVP (i.e., 480.4 ms; 4.7426μ V). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (i.e., 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.",

keywords = "Brain-computer interface (BCI), event-related potential (ERP), gaze-independent, rapid serial visual presentation (RSVP)",

author = "Won, {Dong Ok} and Hwang, {Han Jeong} and Kim, {Dong Min} and M{\"u}ller, {Klaus Robert} and Lee, {Seong Whan}",

note = "Funding Information: Manuscript received May 4, 2016; revised November 7, 2016, March 14, 2017, and July 23, 2017; accepted July 24, 2017. Date of publication August 11, 2017; date of current version February 9, 2018. This work was supported in part by the Institute for Information and Communications Technology Promotion (IITP) through the Korea Government (MSIT) under Grant 2017-0-00451, in part by the Development of BCI-based Brain and Cognitive Computing Technology for Recognizing User{\textquoteright}s Intentions using Deep Learning) and the MSIT, South Korea, under the SW Starlab support program supervised by the IITP under Grant IITP-2015-1107. (Corresponding author: Seong-Whan Lee.) D.-O. Won, D.-M. Kim, and S.-W. Lee are with the Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea (e-mail: [email protected]; [email protected]; [email protected]). Publisher Copyright: {\textcopyright} 2001-2011 IEEE.",

year = "2018",

month = feb,

doi = "10.1109/TNSRE.2017.2736600",

language = "English",

volume = "26",

pages = "334--343",

journal = "IEEE Transactions on Neural Systems and Rehabilitation Engineering",

issn = "1534-4320",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "2",

}

TY - JOUR

T1 - Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces

AU - Won, Dong Ok

AU - Hwang, Han Jeong

AU - Kim, Dong Min

AU - Müller, Klaus Robert

AU - Lee, Seong Whan

N1 - Funding Information: Manuscript received May 4, 2016; revised November 7, 2016, March 14, 2017, and July 23, 2017; accepted July 24, 2017. Date of publication August 11, 2017; date of current version February 9, 2018. This work was supported in part by the Institute for Information and Communications Technology Promotion (IITP) through the Korea Government (MSIT) under Grant 2017-0-00451, in part by the Development of BCI-based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning) and the MSIT, South Korea, under the SW Starlab support program supervised by the IITP under Grant IITP-2015-1107. (Corresponding author: Seong-Whan Lee.) D.-O. Won, D.-M. Kim, and S.-W. Lee are with the Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea (e-mail: [email protected]; [email protected]; [email protected]). Publisher Copyright: © 2001-2011 IEEE.

PY - 2018/2

Y1 - 2018/2

N2 - Most event-related potential (ERP)-based brain-computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (i.e., 360.4-379.6 ms; 5.5867- 5.7662μ V) than the NM-RSVP (i.e., 480.4 ms; 4.7426μ V). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (i.e., 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.

AB - Most event-related potential (ERP)-based brain-computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (i.e., 360.4-379.6 ms; 5.5867- 5.7662μ V) than the NM-RSVP (i.e., 480.4 ms; 4.7426μ V). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (i.e., 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.

KW - Brain-computer interface (BCI)

KW - event-related potential (ERP)

KW - gaze-independent

KW - rapid serial visual presentation (RSVP)

UR - http://www.scopus.com/inward/record.url?scp=85028411397&partnerID=8YFLogxK

U2 - 10.1109/TNSRE.2017.2736600

DO - 10.1109/TNSRE.2017.2736600

M3 - Article

C2 - 28809703

AN - SCOPUS:85028411397

SN - 1534-4320

VL - 26

SP - 334

EP - 343

JO - IEEE Transactions on Neural Systems and Rehabilitation Engineering

JF - IEEE Transactions on Neural Systems and Rehabilitation Engineering

IS - 2

M1 - 8008839

ER -

Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this