TY - JOUR
T1 - Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature
AU - Kim, Sung Soo
AU - Gwak, In Youb
AU - Lee, Seong Whan
N1 - Funding Information:
Manuscript received September 4, 2018; revised March 7, 2019; accepted April 30, 2019. Date of publication June 10, 2019; date of current version May 29, 2020. This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) [No. 2014-0-00059, Development of Predictive Visual Intelligence Technology]. The Associate Editor for this paper was D. F. Wolf. (Sung-Soo Kim and In-Youb Gwak contributed equally to this work.) (Corresponding author: Seong-Whan Lee.) S.-S. Kim is with the Department of Computer and Radio Communications Engineering, Korea University, Seoul 02841, South Korea (e-mail: sungsookim@korea.ac.kr).
Publisher Copyright:
© 2000-2011 IEEE.
PY - 2020/6
Y1 - 2020/6
N2 - The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.
AB - The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.
KW - Advanced driver assistance system
KW - coarse-to-fine learning
KW - continuous orientation estimation
KW - convolutional neural networks
UR - http://www.scopus.com/inward/record.url?scp=85085926336&partnerID=8YFLogxK
U2 - 10.1109/TITS.2019.2919920
DO - 10.1109/TITS.2019.2919920
M3 - Article
AN - SCOPUS:85085926336
SN - 1524-9050
VL - 21
SP - 2522
EP - 2533
JO - IEEE Transactions on Intelligent Transportation Systems
JF - IEEE Transactions on Intelligent Transportation Systems
IS - 6
M1 - 8734128
ER -