A physics-based stochastic framework for activity recognition and analysis

The Neurobiological model of motion recognition posits a Motion Energy Pathway and a Form Pathway but leaves the mechanism for Integration open. In this paper, we present a stochastic Integration methodology, based on the Hamiltonian Monte Carlo, which explores both the Motion and Form space by creating data-driven proposals in the image/form space which are then confirmed in the motion space. We start by using the image (or form) information, which can be shape, texture, colour, or other image properties, to create the proposals. These proposals are then combined with the video tracks to explore the joint motion and form space. To do this, we use a physics-based Hamiltonian via the Hamiltonian Energy Signature to represent a video sequence which is then used in the Hamiltonian Monte Carlo framework to efficiently explore the combined space. Thus, the enormity of the overall search space is reduced by making these more informed proposals. In addition, our framework has potential application to other domains where statistical sampling techniques are useful.