Predicting Decision-Making in the Future: Human Versus Machine

Hoe Sung Ryu, Uijong Ju, Christian Wallraven

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually “understand” the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. In this situation, drivers had to choose between crashing into a suddenly-appeared obstacle or steering their car off a previously indicated cliff. We compared how well humans and a DNN predicted this decision as a function of time before the event. The DNN outperformed humans for early time-points, but had an equal performance for later time-points. Interestingly, spatio-temporal image manipulations and Grad-CAM visualizations uncovered some expected behavior, but also highlighted potential differences in temporal processing for the DNN.

Original languageEnglish
Title of host publicationPattern Recognition - 6th Asian Conference, ACPR 2021, Revised Selected Papers
EditorsChristian Wallraven, Qingshan Liu, Hajime Nagahara
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages15
ISBN (Print)9783031024436
Publication statusPublished - 2022
Event6th Asian Conference on Pattern Recognition, ACPR 2021 - Virtual, Online
Duration: 2021 Nov 92021 Nov 12

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13189 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference6th Asian Conference on Pattern Recognition, ACPR 2021
CityVirtual, Online

Bibliographical note

Funding Information:
Acknowledgements. This work was supported by the National Research Foundation of Korea under Grant NRF-2017M3C7A1041824 and by two Institutes of Information and Communications Technology Planning and Evaluation (IITP) grants funded by the Korean government (MSIT): Development of BCI based Brain and Cognitive Computing Technology for Recognizing User’s Intentions using Deep Learning (2017-0-00451), and Artificial Intelligence Graduate School Program (Korea University) (2019-0-00079).

Publisher Copyright:
© 2022, Springer Nature Switzerland AG.


  • Decision-making
  • Deep learning
  • Humans versus machines
  • Video analysis
  • Video prediction

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Predicting Decision-Making in the Future: Human Versus Machine'. Together they form a unique fingerprint.

Cite this