Improving 3D Occupancy Estimation Using Driver Gaze Estimation

This Camera-only 3D occupancy estimation aims to cost-effectively reconstruct the occupancy state of a grid of voxels in a three-dimensional space, based on input from several cameras. One of the limitations of this approach is detecting occupied voxels of objects located far away, since the cameras resolution at such distances is relatively low. In this work, we boost performance by introducing gaze map estimation. Specifically, we show that although no additional sensor is used, gaze map estimation is strong enough and can be used to enhance basic occupancy estimation networks, yielding better Chamfer distance (CD), F-score and intersection over union (IoU) metrics. At long distances, we found an improvement over the baseline of more than 20% in CD, 24% in F-score and 15% in IoU.