ProtoPLSTM: An Interpretable Deep Learning Approach for Wearable Fine-Grained Fall Detection

Existing models have obtained satisfactory results on whether users have fallen or not. However, in our lives, due to the unpredictability of falls, the binary fall detection models cannot identify different patterns of falls and thus cannot take corresponding protective measures. Therefore, the recognition of more subtle falls has become an urgent challenge. Moreover, the complexity of fall detection models makes it difficult to explain the decision process. In this paper, we propose a new interpretable fine-grained fall detection network, called ProtoPLSTM. The model consists of three main modules: CNN-LSTM encoder backbone network, contextual enhancement module, and ProtoPNet-based network. The first module learns a multi-sensor feature representation through a carefully designed embedding network. The contextual enhancement module expands the receptive field to capture more discriminative features through contextual information and then mines the inter-class differences and intra-class associations of different kinds of falls. The last module globally explains why a fall is detected as some kind of fall from the prototype’s perspective. Our experiments on SisFall, a publicly available dataset with the largest number of fall classes, demonstrate that the proposed method can outperform state-of-the-art methods while having interpretability. The visualization and qualitative analysis explained the important factors and decision-making processes for the occurrence of different kinds of falls from a prototypical perspective.