Contour-based object forecasting for autonomous driving

A novel algorithm, called contour-based object forecasting (COF), to simultaneously perform contour-based segmentation and depth estimation of objects in future frames in autonomous driving systems is proposed in this paper. The proposed algorithm consists of encoding, future forecasting, decoding, and 3D rendering stages. First, we extract the features of observed frames, including past and current frames. Second, from these causal features, we predict the features of future frames using the future forecast module. Third, we decode the predicted features into contour and depth estimates. We obtain object depth maps aligned with segmentation masks via the depth completion using the predicted contours. Finally, from the prediction results, we render the forecasted objects in a 3D space. Experimental results demonstrate that the proposed algorithm reliably forecasts the contours and depths of objects in future frames and that the 3D rendering results intuitively visualize the future locations of the objects.