Monocular 3D Vehicle Trajectory Reconstruction Using Terrain Shape Constraints

This work proposes a novel approach to reconstruct three-dimensional vehicle trajectories in monocular video sequences. We leverage state-of-the-art instance-aware semantic segmentation and optical flow methods to compute object video tracks on pixel level. This approach uses Structure from Motion to determine camera poses relative to vehicle instances and environment structures. We parameterize vehicle trajectories with a single variable by combining object and background reconstructions. The naive combination of vehicle and environment reconstruction results in inconsistent motion trajectories due to the scale ambiguity of SfM. We determine consistent object trajectories by projecting dense vehicle reconstructions on the terrain surface. Our scale ratio estimation approach shows no degenerated camera-vehicle-motions. We demonstrate the usefulness of our approach using publicly available video data of driving scenarios. We extend this evaluation showing trajectory reconstruction results using drone footage. We use synthetic data of vehicles in urban environments to evaluate the proposed algorithm. We achieve an average reconstruction-to-ground-truth distance of 0.17 meter.