Abstract
Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually “understand” the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. In this situation, drivers had to choose between crashing into a suddenly-appeared obstacle or steering their car off a previously indicated cliff. We compared how well humans and a DNN predicted this decision as a function of time before the event. The DNN outperformed humans for early time-points, but had an equal performance for later time-points. Interestingly, spatio-temporal image manipulations and Grad-CAM visualizations uncovered some expected behavior, but also highlighted potential differences in temporal processing for the DNN.
Original language | English |
---|---|
Title of host publication | Pattern Recognition - 6th Asian Conference, ACPR 2021, Revised Selected Papers |
Editors | Christian Wallraven, Qingshan Liu, Hajime Nagahara |
Publisher | Springer Science and Business Media Deutschland GmbH |
Pages | 127-141 |
Number of pages | 15 |
ISBN (Print) | 9783031024436 |
DOIs | |
Publication status | Published - 2022 |
Event | 6th Asian Conference on Pattern Recognition, ACPR 2021 - Virtual, Online Duration: 2021 Nov 9 → 2021 Nov 12 |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 13189 LNCS |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 6th Asian Conference on Pattern Recognition, ACPR 2021 |
---|---|
City | Virtual, Online |
Period | 21/11/9 → 21/11/12 |
Bibliographical note
Funding Information:Acknowledgements. This work was supported by the National Research Foundation of Korea under Grant NRF-2017M3C7A1041824 and by two Institutes of Information and Communications Technology Planning and Evaluation (IITP) grants funded by the Korean government (MSIT): Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning (2017-0-00451), and Artificial Intelligence Graduate School Program (Korea University) (2019-0-00079).
Publisher Copyright:
© 2022, Springer Nature Switzerland AG.
Keywords
- Decision-making
- Deep learning
- Humans versus machines
- Video analysis
- Video prediction
ASJC Scopus subject areas
- Theoretical Computer Science
- General Computer Science