Obstacle Recognition using Computer Vision and Convolutional Neural Networks for Powered Prosthetic Leg Applications

In this work we combine computer vision and a machine learning algorithm, Convolutional Neural Networks (CNNs), to identify obstacles that powered prosthetic leg users might encounter during walking. Our motivation is that powered prosthetic legs could react in synchronicity with their users by recognizing and anticipating the terrain in front of them. We focus on identifying stairs and doors that are within the visual field of a person. To achieve this, we used a compact CNN architecture to optimize image processing for real-time applications. We built and tested a wearable system prototype that included a camera mounted on a pair of glasses and a single-board computer. The prototype was used by able-bodied users to collect and label obstacle and non-obstacle videos, which were used later to train the CNN. In validation, the system was able to recognize around 90% of obstacles across different indoor and outdoor scenarios. The accuracy achieved and the practicality of the prototype shows the potential of computer vision and machine learning in the field of powered prosthetic legs.