Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

Hee Deok Yang; Seong Whan Lee

doi:10.1016/j.patrec.2013.06.022

Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

Hee Deok Yang, Seong Whan Lee

Research output: Contribution to journal › Article › peer-review

40 Citations (Scopus)

Abstract

The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.

Original language	English
Pages (from-to)	2051-2056
Number of pages	6
Journal	Pattern Recognition Letters
Volume	34
Issue number	16
DOIs	https://doi.org/10.1016/j.patrec.2013.06.022
Publication status	Published - 2013

Bibliographical note

Funding Information:
This work was supported by the World Class University Program through the National Research Foundation of Korea funded by the Ministry of Education, Science, and Technology , under Grant R31–10008 . This work was also supported by the National Research Foundation of Korea (NRF) Grant funded by the Korea government (MEST) (No. 2009–0086841 ).

Keywords

BoostMap embedding
Conditional random field
Sign language recognition
Support vector machine

ASJC Scopus subject areas

Software
Signal Processing
Computer Vision and Pattern Recognition
Artificial Intelligence

Access to Document

10.1016/j.patrec.2013.06.022

Cite this

@article{2beb7887ccc84a76a62d35a5a38b6e75,

title = "Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine",

abstract = "The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.",

keywords = "BoostMap embedding, Conditional random field, Sign language recognition, Support vector machine",

author = "Yang, {Hee Deok} and Lee, {Seong Whan}",

note = "Funding Information: This work was supported by the World Class University Program through the National Research Foundation of Korea funded by the Ministry of Education, Science, and Technology , under Grant R31–10008 . This work was also supported by the National Research Foundation of Korea (NRF) Grant funded by the Korea government (MEST) (No. 2009–0086841 ).",

year = "2013",

doi = "10.1016/j.patrec.2013.06.022",

language = "English",

volume = "34",

pages = "2051--2056",

journal = "Pattern Recognition Letters",

issn = "0167-8655",

publisher = "Elsevier B.V.",

number = "16",

}

TY - JOUR

T1 - Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

AU - Yang, Hee Deok

AU - Lee, Seong Whan

N1 - Funding Information: This work was supported by the World Class University Program through the National Research Foundation of Korea funded by the Ministry of Education, Science, and Technology , under Grant R31–10008 . This work was also supported by the National Research Foundation of Korea (NRF) Grant funded by the Korea government (MEST) (No. 2009–0086841 ).

PY - 2013

Y1 - 2013

N2 - The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.

AB - The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.

KW - BoostMap embedding

KW - Conditional random field

KW - Sign language recognition

KW - Support vector machine

UR - http://www.scopus.com/inward/record.url?scp=84883276104&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2013.06.022

DO - 10.1016/j.patrec.2013.06.022

M3 - Article

AN - SCOPUS:84883276104

SN - 0167-8655

VL - 34

SP - 2051

EP - 2056

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

IS - 16

ER -

Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this