Occupancy Flow Fields for Motion Forecasting in Autonomous Driving

Reza Mahjourian, Jinkyu Kim, Yuning Chai, Mingxing Tan, Ben Sapp, Dragomir Anguelov

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)


We propose Occupancy Flow Fields, a new representation for motion forecasting of multiple agents, an important task in autonomous driving.Our representation is a spatio-temporal grid with each grid cell containing both the probability of the cell being occupied by any agent, and a two-dimensional flow vector representing the direction and magnitude of the motion in that cell. Our method successfully mitigates shortcomings of the two most commonly-used representations for motion forecasting: trajectory sets and occupancy grids. Although occupancy grids efficiently represent the probabilistic location of many agents jointly, they do not capture agent motion and lose the agent identities. To this end, we propose a deep learning architecture that generates Occupancy Flow Fields with the help of a new flow trace loss that establishes consistency between the occupancy and flow predictions. We demonstrate the effectiveness of our approach using three metrics on occupancy prediction, motion estimation, and agent ID recovery. In addition, we introduce the problem of predicting speculative agents, which are currently-occluded agents that may appear in the future through dis-occlusion or by entering the field of view. We report experimental results on a large in-house autonomous driving dataset and the public INTERACTION dataset, and show that our model outperforms state-of-the-art models.

Original languageEnglish
Pages (from-to)5639-5646
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number2
Publication statusPublished - 2022 Apr 1

Bibliographical note

Publisher Copyright:
© 2016 IEEE.


  • Autonomous vehicle navigation
  • agent-based systems
  • deep learning methods

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Occupancy Flow Fields for Motion Forecasting in Autonomous Driving'. Together they form a unique fingerprint.

Cite this