Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature

Sung Soo Kim, In Youb Gwak, Seong Whan Lee

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


The continuous orientation estimation of a moving pedestrian is a crucial issue in autonomous driving that requires the detection of a pedestrian intending to cross a road. It is still a challenging task owing to several reasons, including the diversity of pedestrian appearances, the subtle pose difference between adjacent orientations, and similar poses with different orientations such as axisymmetric orientations. These problems render the task highly difficult. Recent studies involving convolutional neural networks (CNNs) have attempted to solve these problems. However, their performance is still far from satisfactory for application in intelligent vehicles. In this paper, we propose a CNN-based two-stream network for continuous orientation estimation. The network can learn representations based on the spatial co-occurrence of visual patterns among pedestrians. To boost estimation performance, we applied a coarse-to-fine learning approach that consists of two learning stages. We investigated continuous orientation performance on the TUD Multiview Pedestrian dataset and the KITTI dataset and compared them with the state-of-the-art methods. The results show that our method outperforms other existing methods.

Original languageEnglish
Article number8734128
Pages (from-to)2522-2533
Number of pages12
JournalIEEE Transactions on Intelligent Transportation Systems
Issue number6
Publication statusPublished - 2020 Jun

Bibliographical note

Funding Information:
Manuscript received September 4, 2018; revised March 7, 2019; accepted April 30, 2019. Date of publication June 10, 2019; date of current version May 29, 2020. This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) [No. 2014-0-00059, Development of Predictive Visual Intelligence Technology]. The Associate Editor for this paper was D. F. Wolf. (Sung-Soo Kim and In-Youb Gwak contributed equally to this work.) (Corresponding author: Seong-Whan Lee.) S.-S. Kim is with the Department of Computer and Radio Communications Engineering, Korea University, Seoul 02841, South Korea (e-mail:

Publisher Copyright:
© 2000-2011 IEEE.


  • Advanced driver assistance system
  • coarse-to-fine learning
  • continuous orientation estimation
  • convolutional neural networks

ASJC Scopus subject areas

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications


Dive into the research topics of 'Coarse-to-Fine Deep Learning of Continuous Pedestrian Orientation Based on Spatial Co-Occurrence Feature'. Together they form a unique fingerprint.

Cite this