Semantic segmentation is one of the most commonly used techniques for road scene understanding. Recently developed deep learning-based semantic segmentation networks are typically based on the encoder-decoder structure and have made great progress in road scene understanding. However, these conventional networks still encounter difficulties in recovering spatial details. To overcome this problem, we introduce a lightweight prediction and boundary-aware refinement module that can hierarchically refine the segmentation results with spatial details. The proposed refinement module has two attention units called the upper-level prediction attention unit and the upper-level boundary attention unit. The upper-level prediction attention unit emphasizes the features in the regions that need to be refined by using predicted class probability from the upper-level, whereas the upper-level boundary attention unit focuses on the features near the semantic boundary of the upper-level segmentation result. By using the proposed prediction and boundary-aware refinement module in the decoder network, the segmentation result can gradually be improved in a top-down manner to a finer and more complete one. Experimental results on the Cityscapes and CamVid datasets demonstrate that the proposed prediction and boundary attention-based refinement module can achieve considerable performance improvement in segmentation accuracy with a marginal increase in computational complexity.
Bibliographical noteFunding Information:
This work was supported by the Ministry of Trade, Industry Energy (MOTIE), South Korea, through the Technology Innovation Program (or Industrial Strategic Technology Development Program) (Development of Deep Learning-Based Open EV Platform Technology Capable of Autonomous Driving) under Grant 10082585.
© 2013 IEEE.
- Attention mechanism
- boundary attention
- deep learning
- feature refinement
- real-time semantic segmentation
- residual learning
- road scene understanding
ASJC Scopus subject areas
- Materials Science(all)
- Electrical and Electronic Engineering
- Computer Science(all)