UGrAD: A graph-theoretic framework for classification of activity with complementary graph boundary detection

Activity recognition and activity boundary detection are two separate long-standing challenges in the image processing literature. In activity recognition, a predefined set of activities is classified using features. Often, subjects do not perform meaningful activities in all the frames, thus requiring the identification of the beginning and the end of the set of contiguous frames containing the activity. This is known as activity boundary detection problem. We seamlessly integrate both tasks by leveraging a single graph-theoretical framework: UGrAD. First, we model the data as a graph and use the spectral properties of its complementary to identify the boundaries of an activity. Next, we introduce the concept of normalized flow between consecutive frames, encoded using a bipartite graph formalism, and use it to capture the temporal evolution of the detected activity. This is accomplished by deriving a robust and reliable feature descriptor from fitting a set of Legendre polynomials to the flow matrix. Lastly, the features are used for activity classification using a linear support vector machine (SVM). Our approach shows substantial improvement over the state-of-the-art activity classification and boundary detection techniques.