Human Pose Prediction Using Interpretable Graph Convolutional Network for Smart Home

Recently, Human Pose Prediction (HPP) using image frames captured by cameras has been widely used in the smart home sector. Combining deep learning vision processing with HPP and using the Graph Convolutional Network (GCN) to extract temporal and spatial features of human actions has achieved satisfactory accuracy. However, there is still a lack of sufficient interpretability to translate theoretical findings into human-centric consumer applications. In this paper, a novel Interpretable GCN-based HPP (IGCN-HPP) model is proposed to address the above problem. Specifically, a multi-layer spatio-temporal convolution is first constructed to capture the depth features in human action data for prediction. Secondly, a GCN Explainer is proposed to assist in model training. When pre-processed graphics frames are fed into the GCN model, it generates various subgraphs. What’s more, using Shapley values from game theory and specific graph rules to assess each subgraph’s contribution to HPP, the neighborhood relationships between subgraphs and among nodes are estimated with the contribution optimization algorithm (COA). The reasonable interpretation of the predicted human pose is obtained by extracting and aggregating the subgraphs that contributed most to the proper prediction category. Qualitative and quantitative experimental results show that the proposed IGCN-HPP outperforms the baseline model in terms of predictive performance.