FPGA-Controlled AI Vision for Prosthetics hand

Implementing vision-based automated control in sophisticated robotics (i.e., prosthetic hands) is limited due to their high-power consumption and computational extensiveness, which may not be achievable through employing commercially available microcontrollers and their coprocessors. Hence, a digital design from scratch and the involvement of the most recent power-efficient but advanced FPGA processors are required. This paper presents the first stage of such an FPGA implementation of an advanced image processing pipeline where an RGB camera is interfaced to a Zynq® UltraScale+™ MPSoC, where an object detection deep learning model is hosted. Acknowledging the complexity of this pipeline, a thorough description of the implementation is provided with a shared public repository. Optimizing the design and circuitry will ensure the implementation of vision sensors to artificially sense an object from the wrist position and facilitate sophisticated grabbing by a prosthetic hand.