Mono-Camera Based Vehicle Orientation Detector for Autonomous Driving

Vehicle orientation estimation is one of the hardest challenges in automotive perception. Point cloud and sensor fusion-based methods are effective, provided there are enough points associated with the target object. This can be challenging for even the most expensive LIDAR sensors over longer distances. In most cases, orientation can be inferred from the movement of an object, however there are edge cases where this does not work such as stationary vehicles, or vehicles that are spinning out of control. Additionally, there are cases, where movement can be predicted from the orientation, which is far more useful. This paper focuses on the development of an extremely light-weight image-based vehicle orientation detector. Previous methods using mono-cameras either jointly estimate orientation, size and position, making the networks large and slow, or are jointly trained with the 2d object detector, decreasing the usability of new object detectors. Our method uses the outputs of a 2D object detector to cut the patches containing objects out of images, and estimate the allocentric orientation of the object in each image patch. This can be transformed into world coordinates, in the case of known camera calibration parameters. This model was trained on a subset of the KITTI dataset and achieves a validation and test accuracy of over 70%. Different techniques for image resizing were evaluated to see whether viewpoint estimation is affected. The method was also tested in real-world conditions at the ZalaZONE proving ground, from a different camera perspective, and different camera parameters compared to the training set.