Probabilistic Modeling of Korean Morphology

Do Gil Lee; Hae Chang Rim

doi:10.1109/TASL.2009.2019922

Probabilistic Modeling of Korean Morphology

Do Gil Lee, Hae Chang Rim

Research output: Contribution to journal › Article › peer-review

20 Citations (Scopus)

Abstract

This paper proposes new probabilistic models for analyzing Korean morphology. In order to take advantage of the characteristics of Korean morphology, the proposed models are based on three linguistic units: eojeol (a Korean spacing unit), morpheme, and syllable. Unlike previous approaches that are based on rules and dictionaries, the probabilistic approach proposed in this study can automatically acquire complete linguistic knowledge from part-of-speech (POS) tagged corpora. In addition, this approach, without any system modification, is easily applicable to other corpora with different tagsets and annotation guidelines. The three different models and their combinations are evaluated on three corpora over a wide range of conditions. The eo-jeol-unit and syllable-unit models compensate for the weaknesses of the morpheme-unit model. The eojeol-unit model performed efficiently, and improved the precision. The syllable-unit model improved in precision as well, showing a particularly robust performance in treating unknown words. The proposed approach is also proven to outperform the previous approaches.

Original language	English
Pages (from-to)	945-955
Number of pages	11
Journal	IEEE Transactions on Audio, Speech and Language Processing
Volume	17
Issue number	5
DOIs	https://doi.org/10.1109/TASL.2009.2019922
Publication status	Published - 2009 Jul

Bibliographical note

Funding Information:
Manuscript received August 01, 2008; revised March 08, 2009. Current version published June 17, 2009. This work was supported in part by a Korea University Grant and in part by the Second Brain Korea 21 Project and WCU (World Class University) program through the Korea Science and Engineering Foundation funded by the Ministry of Education, Science, and Technology (R31-2008-000-10008-0). D.-G. Lee’s Ph.D. dissertation is the basis of the work presented in this paper. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Ruhi Sarikaya.

Keywords

Korean morphology
machine learning
morphologial analysis
probabilistic model

ASJC Scopus subject areas

Acoustics and Ultrasonics
Electrical and Electronic Engineering

Access to Document

10.1109/TASL.2009.2019922

Cite this

@article{ae919b67c4a3475889abc08ab5daf751,

title = "Probabilistic Modeling of Korean Morphology",

abstract = "This paper proposes new probabilistic models for analyzing Korean morphology. In order to take advantage of the characteristics of Korean morphology, the proposed models are based on three linguistic units: eojeol (a Korean spacing unit), morpheme, and syllable. Unlike previous approaches that are based on rules and dictionaries, the probabilistic approach proposed in this study can automatically acquire complete linguistic knowledge from part-of-speech (POS) tagged corpora. In addition, this approach, without any system modification, is easily applicable to other corpora with different tagsets and annotation guidelines. The three different models and their combinations are evaluated on three corpora over a wide range of conditions. The eo-jeol-unit and syllable-unit models compensate for the weaknesses of the morpheme-unit model. The eojeol-unit model performed efficiently, and improved the precision. The syllable-unit model improved in precision as well, showing a particularly robust performance in treating unknown words. The proposed approach is also proven to outperform the previous approaches.",

keywords = "Korean morphology, machine learning, morphologial analysis, probabilistic model",

author = "Lee, {Do Gil} and Rim, {Hae Chang}",

note = "Funding Information: Manuscript received August 01, 2008; revised March 08, 2009. Current version published June 17, 2009. This work was supported in part by a Korea University Grant and in part by the Second Brain Korea 21 Project and WCU (World Class University) program through the Korea Science and Engineering Foundation funded by the Ministry of Education, Science, and Technology (R31-2008-000-10008-0). D.-G. Lee{\textquoteright}s Ph.D. dissertation is the basis of the work presented in this paper. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Ruhi Sarikaya.",

year = "2009",

month = jul,

doi = "10.1109/TASL.2009.2019922",

language = "English",

volume = "17",

pages = "945--955",

journal = "IEEE Transactions on Audio, Speech and Language Processing",

issn = "1558-7916",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "5",

}

TY - JOUR

T1 - Probabilistic Modeling of Korean Morphology

AU - Lee, Do Gil

AU - Rim, Hae Chang

N1 - Funding Information: Manuscript received August 01, 2008; revised March 08, 2009. Current version published June 17, 2009. This work was supported in part by a Korea University Grant and in part by the Second Brain Korea 21 Project and WCU (World Class University) program through the Korea Science and Engineering Foundation funded by the Ministry of Education, Science, and Technology (R31-2008-000-10008-0). D.-G. Lee’s Ph.D. dissertation is the basis of the work presented in this paper. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Ruhi Sarikaya.

PY - 2009/7

Y1 - 2009/7

N2 - This paper proposes new probabilistic models for analyzing Korean morphology. In order to take advantage of the characteristics of Korean morphology, the proposed models are based on three linguistic units: eojeol (a Korean spacing unit), morpheme, and syllable. Unlike previous approaches that are based on rules and dictionaries, the probabilistic approach proposed in this study can automatically acquire complete linguistic knowledge from part-of-speech (POS) tagged corpora. In addition, this approach, without any system modification, is easily applicable to other corpora with different tagsets and annotation guidelines. The three different models and their combinations are evaluated on three corpora over a wide range of conditions. The eo-jeol-unit and syllable-unit models compensate for the weaknesses of the morpheme-unit model. The eojeol-unit model performed efficiently, and improved the precision. The syllable-unit model improved in precision as well, showing a particularly robust performance in treating unknown words. The proposed approach is also proven to outperform the previous approaches.

AB - This paper proposes new probabilistic models for analyzing Korean morphology. In order to take advantage of the characteristics of Korean morphology, the proposed models are based on three linguistic units: eojeol (a Korean spacing unit), morpheme, and syllable. Unlike previous approaches that are based on rules and dictionaries, the probabilistic approach proposed in this study can automatically acquire complete linguistic knowledge from part-of-speech (POS) tagged corpora. In addition, this approach, without any system modification, is easily applicable to other corpora with different tagsets and annotation guidelines. The three different models and their combinations are evaluated on three corpora over a wide range of conditions. The eo-jeol-unit and syllable-unit models compensate for the weaknesses of the morpheme-unit model. The eojeol-unit model performed efficiently, and improved the precision. The syllable-unit model improved in precision as well, showing a particularly robust performance in treating unknown words. The proposed approach is also proven to outperform the previous approaches.

KW - Korean morphology

KW - machine learning

KW - morphologial analysis

KW - probabilistic model

UR - http://www.scopus.com/inward/record.url?scp=85008025475&partnerID=8YFLogxK

U2 - 10.1109/TASL.2009.2019922

DO - 10.1109/TASL.2009.2019922

M3 - Article

AN - SCOPUS:85008025475

SN - 1558-7916

VL - 17

SP - 945

EP - 955

JO - IEEE Transactions on Audio, Speech and Language Processing

JF - IEEE Transactions on Audio, Speech and Language Processing

IS - 5

ER -

Probabilistic Modeling of Korean Morphology

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this