Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature

Sung Soo Kim; In Youb Gwak; Seong Whan Lee

doi:10.1109/TITS.2019.2919920

Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature

Sung Soo Kim, In Youb Gwak, Seong Whan Lee

Department of Artificial Intelligence

Research output: Contribution to journal › Article › peer-review

4 Citations (Scopus)

Abstract

The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.

Original language	English
Article number	8734128
Pages (from-to)	2522-2533
Number of pages	12
Journal	IEEE Transactions on Intelligent Transportation Systems
Volume	21
Issue number	6
DOIs	https://doi.org/10.1109/TITS.2019.2919920
Publication status	Published - 2020 Jun

Bibliographical note

Funding Information:
Manuscript received September 4, 2018; revised March 7, 2019; accepted April 30, 2019. Date of publication June 10, 2019; date of current version May 29, 2020. This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) [No. 2014-0-00059, Development of Predictive Visual Intelligence Technology]. The Associate Editor for this paper was D. F. Wolf. (Sung-Soo Kim and In-Youb Gwak contributed equally to this work.) (Corresponding author: Seong-Whan Lee.) S.-S. Kim is with the Department of Computer and Radio Communications Engineering, Korea University, Seoul 02841, South Korea (e-mail: [email protected]).

Publisher Copyright:
© 2000-2011 IEEE.

Keywords

Advanced driver assistance system
coarse-to-fine learning
continuous orientation estimation
convolutional neural networks

ASJC Scopus subject areas

Automotive Engineering
Mechanical Engineering
Computer Science Applications

Access to Document

10.1109/TITS.2019.2919920

Cite this

@article{f7682596de534e44874417bbbd1ce417,

title = "Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature",

abstract = "The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.",

keywords = "Advanced driver assistance system, coarse-to-fine learning, continuous orientation estimation, convolutional neural networks",

author = "Kim, {Sung Soo} and Gwak, {In Youb} and Lee, {Seong Whan}",

note = "Funding Information: Manuscript received September 4, 2018; revised March 7, 2019; accepted April 30, 2019. Date of publication June 10, 2019; date of current version May 29, 2020. This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) [No. 2014-0-00059, Development of Predictive Visual Intelligence Technology]. The Associate Editor for this paper was D. F. Wolf. (Sung-Soo Kim and In-Youb Gwak contributed equally to this work.) (Corresponding author: Seong-Whan Lee.) S.-S. Kim is with the Department of Computer and Radio Communications Engineering, Korea University, Seoul 02841, South Korea (e-mail: [email protected]). Publisher Copyright: {\textcopyright} 2000-2011 IEEE.",

year = "2020",

month = jun,

doi = "10.1109/TITS.2019.2919920",

language = "English",

volume = "21",

pages = "2522--2533",

journal = "IEEE Transactions on Intelligent Transportation Systems",

issn = "1524-9050",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "6",

}

TY - JOUR

T1 - Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature

AU - Kim, Sung Soo

AU - Gwak, In Youb

AU - Lee, Seong Whan

N1 - Funding Information: Manuscript received September 4, 2018; revised March 7, 2019; accepted April 30, 2019. Date of publication June 10, 2019; date of current version May 29, 2020. This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) [No. 2014-0-00059, Development of Predictive Visual Intelligence Technology]. The Associate Editor for this paper was D. F. Wolf. (Sung-Soo Kim and In-Youb Gwak contributed equally to this work.) (Corresponding author: Seong-Whan Lee.) S.-S. Kim is with the Department of Computer and Radio Communications Engineering, Korea University, Seoul 02841, South Korea (e-mail: [email protected]). Publisher Copyright: © 2000-2011 IEEE.

PY - 2020/6

Y1 - 2020/6

N2 - The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.

AB - The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.

KW - Advanced driver assistance system

KW - coarse-to-fine learning

KW - continuous orientation estimation

KW - convolutional neural networks

UR - http://www.scopus.com/inward/record.url?scp=85085926336&partnerID=8YFLogxK

U2 - 10.1109/TITS.2019.2919920

DO - 10.1109/TITS.2019.2919920

M3 - Article

AN - SCOPUS:85085926336

SN - 1524-9050

VL - 21

SP - 2522

EP - 2533

JO - IEEE Transactions on Intelligent Transportation Systems

JF - IEEE Transactions on Intelligent Transportation Systems

IS - 6

M1 - 8734128

ER -

Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this