Predicting Driver Attention in Critical Situations

Ye Xia, Danqing Zhang, Jinkyu Kim, Ken Nakayama, Karl Zipser, David Whitney

Research output: Chapter in Book/Report/Conference proceedingConference contribution

59 Citations (Scopus)


Robust driver attention prediction for critical situations is a challenging computer vision problem, yet essential for autonomous driving. Because critical driving moments are so rare, collecting enough data for these situations is difficult with the conventional in-car data collection protocol—tracking eye movements during driving. Here, we first propose a new in-lab driver attention collection protocol and introduce a new driver attention dataset, Berkeley DeepDrive Attention (BDD-A) dataset, which is built upon braking event videos selected from a large-scale, crowd-sourced driving video dataset. We further propose Human Weighted Sampling (HWS) method, which uses human gaze behavior to identify crucial frames of a driving dataset and weights them heavily during model training. With our dataset and HWS, we built a driver attention prediction model that outperforms the state-of-the-art and demonstrates sophisticated behaviors, like attending to crossing pedestrians but not giving false alarms to pedestrians safely walking on the sidewalk. Its prediction results are nearly indistinguishable from ground-truth to humans. Although only being trained with our in-lab attention data, the model also predicts in-car driver attention data of routine driving with state-of-the-art accuracy. This result not only demonstrates the performance of our model but also proves the validity and usefulness of our dataset and data collection protocol.

Original languageEnglish
Title of host publicationComputer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
EditorsHongdong Li, C.V. Jawahar, Konrad Schindler, Greg Mori
PublisherSpringer Verlag
Number of pages17
ISBN (Print)9783030208721
Publication statusPublished - 2019
Externally publishedYes
Event14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia
Duration: 2018 Dec 22018 Dec 6

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11365 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference14th Asian Conference on Computer Vision, ACCV 2018


  • BDD-A dataset
  • Berkeley DeepDrive
  • Driver attention prediction

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Predicting Driver Attention in Critical Situations'. Together they form a unique fingerprint.

Cite this