Recent natural language understanding (NLU) research on the Korean language has been vigorously maturing with the advancements of pretrained language models and datasets. However, Korean pretrained language models still struggle to generate a short sentence with a given condition based on compositionality and commonsense reasoning (i.e., generative commonsense reasoning). The two major challenges are inadequate data resources to develop generative commonsense reasoning regarding Korean linguistic features and to evaluate language models which are necessary for natural language generation (NLG). To solve these problems, we propose a textgeneration dataset for Korean generative commonsense reasoning and language model evaluation. In this work, a semi-automatic dataset construction approach filters out contents inexplicable to commonsense, ascertains quality, and reduces the cost of building the dataset. We also present an in-depth analysis of the generation results of language models with various evaluation metrics along with human-annotated scores. The whole dataset is publicly available at (https://aihub.or. kr/opendata/korea-university).
|Title of host publication||Findings of the Association for Computational Linguistics|
|Subtitle of host publication||NAACL 2022 - Findings|
|Publisher||Association for Computational Linguistics (ACL)|
|Number of pages||17|
|Publication status||Published - 2022|
|Event||2022 Findings of the Association for Computational Linguistics: NAACL 2022 - Seattle, United States|
Duration: 2022 Jul 10 → 2022 Jul 15
|Name||Findings of the Association for Computational Linguistics: NAACL 2022 - Findings|
|Conference||2022 Findings of the Association for Computational Linguistics: NAACL 2022|
|Period||22/7/10 → 22/7/15|
Bibliographical noteFunding Information:
This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2018-0-01405) supervised by the IITP(Institute for Information & Communications Technology Planning & Evaluation). This work was supported by Institute of Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques). This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(NRF-2021R1A6A1A03045425).
© Findings of the Association for Computational Linguistics: NAACL 2022 - Findings.
ASJC Scopus subject areas
- Computational Theory and Mathematics
- Computer Science Applications
- Information Systems