A Privacy-Preserving AIoT Framework for Fall Detection and Classification Using Hierarchical Learning With Multilevel Feature Fusion

Privacy and false fall detection pose a significant challenge within the current camera-based human activity monitoring research. In response, we propose a solution that leverages the inherent relationship between binary fall detection and activity classification through hierarchical learning for low-false fall classification. Our solution involves employing edge computing for data preprocessing, with a specific focus on extracting key points of the human body and motion features. Subsequently, the classification process based on multistream hierarchical learning takes place at the cloud level. This approach prevents sending raw RGB videos to the cloud to improve data privacy. Moreover, our network incorporates the hierarchical relationship between binary and multiclass classification using stream-to-stream skip connections and multilevel feature fusion. We created our multiscale convolutional fusion block (MSCFB) for multilevel feature extraction and fusion with an inner block skip connection. Additionally, we added an auxiliary fall detection loss to the first stream to regulate and control the feature extraction process for fall detection. Our network attained state-of-the-art performance in activity classification on the UP-Fall data set, securing an F1-score of 97.27%. Notably, there were no misclassifications between regular activities and fall subclasses, leading to a perfect F1-score of 100% for binary fall detection on this data set. Furthermore, our fall detection network achieved state-of-the-art on the PRECIS HAR data set with an F1-score of 98.34%.