Privacy-Preserving Human Activity Recognition System for Assisted Living Environments

Automatic human activity recognition has numerous applications, especially in elderly support and healthcare. Several approaches for human activity recognition using a variety of sensors are available in the literature. While such frameworks are effective, each has limitations related to privacy, convenience, cost, and performance. In this paper, a robust framework for automatic human activity recognition is proposed that uses depth sensors that preserve privacy and are cost-effective. The depth sensors provide two data modalities, namely depth maps and skeleton sequences, used together for activity recognition. Two novel descriptors, Joint Position Descriptor (JPD) based on the position of joints; and Bone Angle Descriptor (BAD) based on bone inclination, are generated from the skeleton sequence data. The descriptors convey both spatial and temporal information and are scale and view-point invariant. Depth video clips are used along with the descriptors to deal with the issue of noisy and missing skeleton sequences. The data modalities and descriptors are fused using a two-level fusion strategy for a multi-channel Convolutional Neural Network (CNN) framework. The proposed system is validated and shown to be superior to the existing state of the art through comparisons over four widely used public datasets. A computational complexity analysis of the system confirms its efficacy in real time. A prototypical implementation of the proposed system further validates its practicability.