AdaHPE: Adaptive Human Pose Estimation on Resource-Constrained Edge Computing Devices via Temporal Propagation

This article presents AdaHPE, an innovative and efficient framework for human pose estimation (HPE) designed specifically for edge computing devices with constrained and fluctuating resources. AdaHPE redefines the conventional HPE workflow by converting the resource-demanding pose regression into a sequence of computationally feasible pose propagation tasks. The framework incorporates a memory-augmented long short-term memory (LSTM) network with a global memory repository, allowing AdaHPE to adaptively choose keyframes based on real-time data and the device’s resource status, thereby optimizing the tradeoff between accuracy and computational efficiency. A reinforcement learning component is further integrated to intelligently adjust the ratio of keyframes used, enhancing the framework’s adaptability. Utilizing policy gradient algorithms, AdaHPE is optimized to maximize a reward function that encourages both accurate and resource-efficient pose estimations, while respecting a given keyframe constraint. Extensive experiments on benchmarks, including Penn Action, Sub-JHMDB, NTU RGB+D 120, and real-world datasets demonstrate that AdaHPE can significantly reduce computational overhead compared to per-frame HPE models while preserving high accuracy and robustness under varying resource limitations. Moreover, the seamless compatibility of our approach with various off-the-shelf HPE models highlights its versatility and potential for broad applications.