TY - GEN
T1 - Predicting Decision-Making in the Future
T2 - 6th Asian Conference on Pattern Recognition, ACPR 2021
AU - Ryu, Hoe Sung
AU - Ju, Uijong
AU - Wallraven, Christian
N1 - Funding Information:
Acknowledgements. This work was supported by the National Research Foundation of Korea under Grant NRF-2017M3C7A1041824 and by two Institutes of Information and Communications Technology Planning and Evaluation (IITP) grants funded by the Korean government (MSIT): Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning (2017-0-00451), and Artificial Intelligence Graduate School Program (Korea University) (2019-0-00079).
Publisher Copyright:
© 2022, Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually “understand” the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. In this situation, drivers had to choose between crashing into a suddenly-appeared obstacle or steering their car off a previously indicated cliff. We compared how well humans and a DNN predicted this decision as a function of time before the event. The DNN outperformed humans for early time-points, but had an equal performance for later time-points. Interestingly, spatio-temporal image manipulations and Grad-CAM visualizations uncovered some expected behavior, but also highlighted potential differences in temporal processing for the DNN.
AB - Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually “understand” the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. In this situation, drivers had to choose between crashing into a suddenly-appeared obstacle or steering their car off a previously indicated cliff. We compared how well humans and a DNN predicted this decision as a function of time before the event. The DNN outperformed humans for early time-points, but had an equal performance for later time-points. Interestingly, spatio-temporal image manipulations and Grad-CAM visualizations uncovered some expected behavior, but also highlighted potential differences in temporal processing for the DNN.
KW - Decision-making
KW - Deep learning
KW - Humans versus machines
KW - Video analysis
KW - Video prediction
UR - http://www.scopus.com/inward/record.url?scp=85130252204&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-02444-3_10
DO - 10.1007/978-3-031-02444-3_10
M3 - Conference contribution
AN - SCOPUS:85130252204
SN - 9783031024436
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 127
EP - 141
BT - Pattern Recognition - 6th Asian Conference, ACPR 2021, Revised Selected Papers
A2 - Wallraven, Christian
A2 - Liu, Qingshan
A2 - Nagahara, Hajime
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 9 November 2021 through 12 November 2021
ER -