Kalman filtering aspects in camera and deep learning based tracking for traffic monitoring

In multiple object tracking applications for traffic monitoring the underlying algorithms often use rectangular, axis-aligned bounding boxes from deep-learning based object detection systems as a measurement input. Often the association of the measurements to trajectories is performed in the image domain, where after for every bounding box an already associated pseudo-measurement in a world coordinate system is estimated, which is finally used as a measurement input to a Kalman Filter. In contrast to this approach this article examines a multiple object tracking system with a measurement model which maps the estimated state of objects in world coordinates to the aforementioned rectangular bounding boxes in an image coordinate system. In addition the choice of the state vector elements modelled to represent the vehicles is shown and discussed. The approach presented in this article allows for association founded in physical reality, the estimation of the spacial dimensions of tracked objects and avoids shortcomings of a two-staged approach with association in the image coordinate frame.