A Single Stage Multitasking Model for Person Detection and Activity Recognition

The integration of computer vision technologies into our daily lives has become increasingly prevalent, finding applications in diverse fields such as surveillance and public safety. A crucial aspect of these applications involves the detection of individuals, tracking their movements, and recognizing their activities. Traditional approaches often employ separate specialised models for each of these tasks, resulting in complex video processing pipelines that may strain computational resources, particularly in scenarios where real-time analytics is imperative. This work explores single stage multitasking architecture for person detection and activity recognition, without any explicit tracking, segmentation or body keypoint estimation.Real-time action recognition is a challenging task. Online/realtime models cannot leverage information from future frames, hence it cannot perform equally well like the offline/non-realtime models. The actions may be incomplete or look very similar at early stages. The actions can be of varying duration, and multiple people will be performing same or different actions in a video. Despite these challenges, the proposed model achieved 19.17mAP on the AVA dataset $\mathbf{v}.2$ at 36 frames per second (FPS) and an optimised version runs at 51 FPS with the same accuracy levels on an RTX 3090 GPU. We were able to optimise the video processing pipeline by eliminating the need for multiple specialized models.