Towards Specialized Hardware for Learning-based Visual Odometry on the Edge

Learning-based visual odometry (VO) has gained increasing popularity in autonomous navigation of small robots. However, most methods in the category require computation resources not normally available on edge systems. We contend that specialized hardware accelerators are ideal solutions to this problem because of their superior energy efficiency. In this paper, we first propose a model to derive compute specifications for VO from physical characteristics of unmanned aerial vehicles (UAVs). These specifications serve as the basis to guide our accelerator design process. Based on the specifications derived from the DJI Mavic Air 2 and Crazyflie 2.0 UAVs, we explore the speed/flight-time design spaces for three target VO algorithms on two NVIDIA Jetson systems. Then, we propose a hardware accelerator architecture and present prototype implementations based on FPGAs. Additionally, we illustrate the algorithm/hardware co-design approach with a series of hardware-aware algorithmic redesigns targeting the FPGA prototypes, and quantify the throughput-accuracy tradeoff of them. Our FPGA implementation of DFVO is 2.7x more energy efficient compared to off-the-shelf embedded computers.