Attention-Inspired Path Prediction and Adaptive Obstacle Perception Architecture for Powered Lower-Limb Prostheses

Path prediction and obstacle perception are crucial intelligence capabilities for powered lower-limb prostheses (PLLPs). These capabilities enable the devices to understand the amputee’s movement intentions and the walking environment, thereby enhancing the navigation abilities of PLLPs. However, it is challenging for PLLPs to achieve ideal navigation performance by relying solely on force/motion sensors. The human vision system can quickly perceive and focus on obstacles during locomotion and conduct obstacle analysis, demonstrating the potential application of vision sensors in PLLPs’ navigation. This article reports an attention-inspired path prediction and adaptive obstacle perception architecture (AI-PPOPA). The architecture consists of four modules: visual odometry, object detection, path prediction, and obstacle information estimation, which enables PLLPs to realize path prediction, obstacle perception and information estimation, and real-time obstacle localization. We deployed the system on the Nvidia Jetson AGX Orin for experimental testing. The experimental results indicate that the RMSE for the distance estimation accuracy for small obstacles ranges from as high as $2.81~\pm ~1.06$ cm to as low as $4.98~\pm ~1.58$ cm, while for medium obstacles, it ranges from as high as $4.62~\pm ~1.11$ cm to as low as $5.46~\pm ~3.45$ cm. Additionally, processing images at 84 ms per image demonstrates its capability for real-time control and improves the adaptability of PLLPs in complex terrain, allowing the amputee to walk more safely.