K-NCT: Korean Neural Grammatical Error Correction Gold-Standard Test Set Using Novel Error Type Classification Criteria

Seonmin Koo; Chanjun Park; Jaehyung Seo; Seungjun Lee; Hyeonseok Moon; Jungseob Lee; Heuiseok Lim

doi:10.1109/ACCESS.2022.3219448

K-NCT: Korean Neural Grammatical Error Correction Gold-Standard Test Set Using Novel Error Type Classification Criteria

Seonmin Koo, Chanjun Park, Jaehyung Seo, Seungjun Lee, Hyeonseok Moon, Jungseob Lee, Heuiseok Lim

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

Abstract

Recently, active research has been conducted on Korean grammatical error correction on machine translation (MT) and automatic noise generation. However, there is no gold-standard test set for objective and official comparative analysis. A significant limitation is measuring the ill-defined performance because the experimental error types in the train set are also included in the test set. Moreover, error types in the training set are also included in the test set. Additionally, the types of errors for qualitative analysis are defined differently with no explicit guidelines. This study proposes a gold-standard test set called the Korean Neural Grammatical Correction Test set (K-NCT) for Korean grammatical error correction using a new error type classification guideline. To ensure the factuality and reliability of the proposal, we conduct a quantitative analysis using a commercialization system and human evaluation. Experimental results demonstrate that the proposed grammatical error correction test set has a well-balanced, diverse, and precise guideline. Our dataset is available at https://github.com/seonminkoo/K-NCT

Original language	English
Pages (from-to)	118167-118175
Number of pages	9
Journal	IEEE Access
Volume	10
DOIs	https://doi.org/10.1109/ACCESS.2022.3219448
Publication status	Published - 2022

Bibliographical note

Funding Information:
This work was supported in part by the Ministry of Science and ICT, South Korea, under the Information Technology Research Center Support Program supervised by the Institute for Information and Communications Technology Planning and Evaluation, under Grant IITP-2018-0-01405; and in part by the Basic Science Research Program through the National Research Foundation of Korea, Ministry of Education, under Grant NRF-2022R1A2C1007616.

Publisher Copyright:
© 2013 IEEE.

Keywords

Korean grammar correction
error standard
gold test set
human evaluation

ASJC Scopus subject areas

General Engineering
General Materials Science
General Computer Science

Access to Document

10.1109/ACCESS.2022.3219448

Cite this

@article{7ccd57e5d29a415b8f6c15cc024beb62,

title = "K-NCT: Korean Neural Grammatical Error Correction Gold-Standard Test Set Using Novel Error Type Classification Criteria",

abstract = "Recently, active research has been conducted on Korean grammatical error correction on machine translation (MT) and automatic noise generation. However, there is no gold-standard test set for objective and official comparative analysis. A significant limitation is measuring the ill-defined performance because the experimental error types in the train set are also included in the test set. Moreover, error types in the training set are also included in the test set. Additionally, the types of errors for qualitative analysis are defined differently with no explicit guidelines. This study proposes a gold-standard test set called the Korean Neural Grammatical Correction Test set (K-NCT) for Korean grammatical error correction using a new error type classification guideline. To ensure the factuality and reliability of the proposal, we conduct a quantitative analysis using a commercialization system and human evaluation. Experimental results demonstrate that the proposed grammatical error correction test set has a well-balanced, diverse, and precise guideline. Our dataset is available at https://github.com/seonminkoo/K-NCT",

keywords = "Korean grammar correction, error standard, gold test set, human evaluation",

author = "Seonmin Koo and Chanjun Park and Jaehyung Seo and Seungjun Lee and Hyeonseok Moon and Jungseob Lee and Heuiseok Lim",

note = "Funding Information: This work was supported in part by the Ministry of Science and ICT, South Korea, under the Information Technology Research Center Support Program supervised by the Institute for Information and Communications Technology Planning and Evaluation, under Grant IITP-2018-0-01405; and in part by the Basic Science Research Program through the National Research Foundation of Korea, Ministry of Education, under Grant NRF-2022R1A2C1007616. Publisher Copyright: {\textcopyright} 2013 IEEE.",

year = "2022",

doi = "10.1109/ACCESS.2022.3219448",

language = "English",

volume = "10",

pages = "118167--118175",

journal = "IEEE Access",

issn = "2169-3536",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - K-NCT

T2 - Korean Neural Grammatical Error Correction Gold-Standard Test Set Using Novel Error Type Classification Criteria

AU - Koo, Seonmin

AU - Park, Chanjun

AU - Seo, Jaehyung

AU - Lee, Seungjun

AU - Moon, Hyeonseok

AU - Lee, Jungseob

AU - Lim, Heuiseok

N1 - Funding Information: This work was supported in part by the Ministry of Science and ICT, South Korea, under the Information Technology Research Center Support Program supervised by the Institute for Information and Communications Technology Planning and Evaluation, under Grant IITP-2018-0-01405; and in part by the Basic Science Research Program through the National Research Foundation of Korea, Ministry of Education, under Grant NRF-2022R1A2C1007616. Publisher Copyright: © 2013 IEEE.

PY - 2022

Y1 - 2022

N2 - Recently, active research has been conducted on Korean grammatical error correction on machine translation (MT) and automatic noise generation. However, there is no gold-standard test set for objective and official comparative analysis. A significant limitation is measuring the ill-defined performance because the experimental error types in the train set are also included in the test set. Moreover, error types in the training set are also included in the test set. Additionally, the types of errors for qualitative analysis are defined differently with no explicit guidelines. This study proposes a gold-standard test set called the Korean Neural Grammatical Correction Test set (K-NCT) for Korean grammatical error correction using a new error type classification guideline. To ensure the factuality and reliability of the proposal, we conduct a quantitative analysis using a commercialization system and human evaluation. Experimental results demonstrate that the proposed grammatical error correction test set has a well-balanced, diverse, and precise guideline. Our dataset is available at https://github.com/seonminkoo/K-NCT

AB - Recently, active research has been conducted on Korean grammatical error correction on machine translation (MT) and automatic noise generation. However, there is no gold-standard test set for objective and official comparative analysis. A significant limitation is measuring the ill-defined performance because the experimental error types in the train set are also included in the test set. Moreover, error types in the training set are also included in the test set. Additionally, the types of errors for qualitative analysis are defined differently with no explicit guidelines. This study proposes a gold-standard test set called the Korean Neural Grammatical Correction Test set (K-NCT) for Korean grammatical error correction using a new error type classification guideline. To ensure the factuality and reliability of the proposal, we conduct a quantitative analysis using a commercialization system and human evaluation. Experimental results demonstrate that the proposed grammatical error correction test set has a well-balanced, diverse, and precise guideline. Our dataset is available at https://github.com/seonminkoo/K-NCT

KW - Korean grammar correction

KW - error standard

KW - gold test set

KW - human evaluation

UR - http://www.scopus.com/inward/record.url?scp=85141616730&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2022.3219448

DO - 10.1109/ACCESS.2022.3219448

M3 - Article

AN - SCOPUS:85141616730

SN - 2169-3536

VL - 10

SP - 118167

EP - 118175

JO - IEEE Access

JF - IEEE Access

ER -

K-NCT: Korean Neural Grammatical Error Correction Gold-Standard Test Set Using Novel Error Type Classification Criteria

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this