Failure detection for semantic segmentation on road scenes using deep learning

Junho Song, Woojin Ahn, Sangkyoo Park, Myotaeg Lim

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)


Detecting failure cases is an essential element for ensuring the safety self-driving system. Any fault in the system directly leads to an accident. In this paper, we analyze the failure of semantic segmentation, which is crucial for autonomous driving system, and detect the failure cases of the predicted segmentation map by predicting mean intersection of union (mIoU). Furthermore, we design a deep neural network for predicting mIoU of segmentation map without the ground truth and introduce a new loss function for training imbalance data. The proposed method not only predicts the mIoU, but also detects failure cases using the predicted mIoU value. The experimental results on Cityscapes data show our network gives prediction accuracy of 93.21% and failure detection accuracy of 84.8% . It also performs well on a challenging dataset generated from the vertical vehicle camera of the Hyundai Motor Group with 90.51% mIoU prediction accuracy and 83.33% failure detection accuracy.

Original languageEnglish
Article number1870
Pages (from-to)1-22
Number of pages22
JournalApplied Sciences (Switzerland)
Issue number4
Publication statusPublished - 2021 Feb 2

Bibliographical note

Funding Information:
This research was supported by the Hyundai Motor Group (HMG) funded by the Hyundai NGV and in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) (Grants No. NRF-2016R1D1A1B01016071).

Publisher Copyright:
© 2021 by the authors. Licensee MDPI, Basel, Switzerland.


  • Autonomous driving system
  • Convolutional neural network (CNN)
  • Failure detection
  • Semantic segmentation

ASJC Scopus subject areas

  • General Materials Science
  • Instrumentation
  • General Engineering
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes


Dive into the research topics of 'Failure detection for semantic segmentation on road scenes using deep learning'. Together they form a unique fingerprint.

Cite this