Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

Hee Deok Yang, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

38 Citations (Scopus)

Abstract

The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.

Original languageEnglish
Pages (from-to)2051-2056
Number of pages6
JournalPattern Recognition Letters
Volume34
Issue number16
DOIs
Publication statusPublished - 2013

Bibliographical note

Funding Information:
This work was supported by the World Class University Program through the National Research Foundation of Korea funded by the Ministry of Education, Science, and Technology , under Grant R31–10008 . This work was also supported by the National Research Foundation of Korea (NRF) Grant funded by the Korea government (MEST) (No. 2009–0086841 ).

Keywords

  • BoostMap embedding
  • Conditional random field
  • Sign language recognition
  • Support vector machine

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine'. Together they form a unique fingerprint.

Cite this