Robust facial landmark extraction scheme using multiple convolutional neural networks

Hyungjoon Kim, Jisoo Park, Hyeon Woo Kim, Eenjun Hwang, Seungmin Rho

    Research output: Contribution to journalArticlepeer-review

    9 Citations (Scopus)

    Abstract

    Facial landmarks are a set of features that can be distinguished on the human face with the naked eye. Typical facial landmarks include eyes, eyebrows, nose, and mouth. Landmarks play an important role in human-related image analysis. For example, they can be used to determine whether there is a human being in the image, identify who the person is, or recognize the orientation of a face when taking a photograph. General techniques for detecting facial landmarks can be classified into two groups: One is based on traditional image processing techniques, such as Haar cascade classifiers and edge detection. The other is based on machine learning techniques in which landmarks can be detected by training neural network using facial features. However, such techniques have shown low accuracy, especially in some special conditions such as low luminance and overlapped faces. To overcome these problems, we proposed in our previous work a facial landmark extraction scheme using deep learning and semantic segmentation, and demonstrated that with even a small dataset, our scheme could achieve reasonable facial landmark extraction performance under such conditions. Nevertheless, for more extensive dataset, we found several exceptional cases where the scheme could not detect face landmarks precisely. Hence, in this paper, we revise our facial landmark extraction scheme using a deep learning model called Faster R-CNN and show how our scheme can improve the overall performance by handling such exceptional cases appropriately. Also, we show how to expand the training dataset by using image filters and image operations such as rotation for more robust landmark detection.

    Original languageEnglish
    Pages (from-to)3221-3238
    Number of pages18
    JournalMultimedia Tools and Applications
    Volume78
    Issue number3
    DOIs
    Publication statusPublished - 2019 Feb 1

    Bibliographical note

    Funding Information:
    Acknowledgements This work was supported by Korea Environment Industry & Technology Institute (KEITI) through Public Technology Program based on Environmental Policy, funded by Korea Ministry of Environment (MOE)(2017000210001).

    Funding Information:
    This work was supported by Korea Environment Industry & Technology Institute (KEITI) through Public Technology Program based on Environmental Policy, funded by Korea Ministry of Environment (MOE)(2017000210001).

    Publisher Copyright:
    © 2018, Springer Science+Business Media, LLC, part of Springer Nature.

    Keywords

    • Convolutional neural networks
    • Facial landmark
    • Faster R-CNN
    • Object detection
    • Semantic segmentation

    ASJC Scopus subject areas

    • Software
    • Media Technology
    • Hardware and Architecture
    • Computer Networks and Communications

    Fingerprint

    Dive into the research topics of 'Robust facial landmark extraction scheme using multiple convolutional neural networks'. Together they form a unique fingerprint.

    Cite this