FPGA-Accelerated Sim-to-Real Control Policy Learning for Robotic Arms

Sim-to-real robot learning has been used in various applications, but its implementation in software may not provide the best performance. This tutorial describes how hardware acceleration based on Field-Programmable Gate Array (FPGA) technology for deep reinforcement learning can improve sim-to-real robot control policy learning. A novel architecture for the Deep Deterministic Policy Gradient (DDPG) algorithm is developed for a full-stack sim-to-real development platform to learn control policies for robotic arms. The capability of our development platform is illustrated by transferring learned policies encoded as fixed-point numbers from our implementation to a miniature robotic arm.