Foot Placement Prediction for Assistive Walking by Fusing Sequential 3D Gaze and Environmental Context

Predicting the locomotion intent of humans is important for controlling assistive robots. Previous studies have investigated assistive walking on structured terrains, but only a few studies have considered rough terrains. Human intent on rough terrains is more difficult to predict because there is a transition at every step. To predict the foot placements of humans on rough terrains, the present paper fuses sequential 3D gaze and the environmental context. The 3D gaze is assumed to be the intersection point of the line of sight as measured by an eye-tracker and the environmental point cloud as measured by an RGBD camera. The sequential 3D gaze and the environmental context are fused based on an RGBD SLAM algorithm. Then the segmented terrain that is closest to the center of sequential 3D gaze is regarded as the most possible foothold area at the next step. Six able-bodied subjects are invited to walk randomly on rough terrains. Their foot placements are labeled and compared with the predicted foot placements. Experimental results show that the proposed method can predict the foot placements of all subjects 0.5 step ahead. With environmental context and user-dependent time window, the distance error of predicting the foot placements can decrease to 0.086 m. Hence, gaze, environmental context, and time window are all important in predicting the human intent when navigating rough terrains.