Quaternion-driven CNN for Object Pose Tracking

In this work we propose a new mixed-input neural network for instance level monocular object tracking. The proposed Y-like architecture has two inputs and one output. An object id with a quaternion representing the object rotation in the previous frame are fed to the first input, whereas the object sub-window is fed to the second input. We demonstrate that on the basis of quaternions the neural network learns attention blobs representing the object rotation in the previous frame. A single neural network has been trained for six objects to estimate their fiducial points in sequences of RGB images. A tracking by optimization approach has been leveraged in the experiments. The algorithm has been evaluated on the OPT benchmark dataset for 6DoF object pose tracking as well as a custom dataset including image sequences with both real and rendered objects.