Back to Old Constraints to Jointly Supervise Learning Depth, Camera Motion and Optical Flow in a Monocular Video

In structure from motion or similarly in monocular SLAM problems, spatio-temporal image variations, motion and scene geometry are intimately related and in the absence of such a constraint, unsupervised deep learning methods often tend to state the problem under multiple constraints. We readdress the problem of 3D interpretation estimation as an unsupervised deep learning process where depth and camera motion are learned to satisfy the 3D brightness constraint for rigid objects. We introduce for the first time a new learning paradigm where the spatio-temporal variations of image sequences are coupled to 3D interpretation to minimize the loss without need to add more ad-hoc constraints that are not related to the 3D interpretation. Experimental results show that our method competes and sometimes outperforms the state-of-the-art methods.