Multiple Object Association Incorporating Object Tracking, Depth, and Velocity Analysis on 2D Videos

In today's world of large-scale, rapid data creation, using analytic algorithms to generate knowledge from that data is of high value. While many datasets have robust proven analytics, generating knowledge from video has been a persistent challenge. Various methods exist to extract useful information from a scene in an image or video, such as object detection, object classifiers, and, recently, human-object interaction (HOI), which attempts to identify and describe interactions between humans and objects. Although the field of HOI has made excellent strides, there are still significant challenges associated with HOI tasks, such as availability of datasets and the type and diversity of videos on which it will work. In this paper, we describe a system that is able to solve a relaxed version of the HOI problem, where only the presence of an interaction between humans and objects is of interest, rather than the type of interaction itself. Our system combines general, off-the-shelf methods of object detection, object tracking, monocular depth estimation, and velocity calculations to analyze video data and report the presence of multiple object associations. We find that this information is of great value in providing search and discovery in different use cases such as surveillance, security, data labelling, and advanced analytics.