A High-Performance ORB Accelerator with Algorithm and Hardware Co-design for Visual Localization

Vision-based localization plays an important role in numerous emerging applications, including mobile robots, UAVs, AR/VR, etc. The ORB algorithm is a very classic algorithm in visual localization, typically consisting of feature point extraction and matching. Owing to the extensive computational load involved in these processes, real-time performance remains difficult in applications based on embedded devices. In this work, we present a high-performance ORB accelerator through the algorithm and hardware co-design, including a three-levels parallel processing architecture to improve performance, a pixel-level RS-BRIEF descriptor generator to reduce the computational complexity, and an on-chip sliding storage bucket matching and overwriting strategy for storage optimization. The proposed work has been implemented on the ZCU104 FPGA board achieving a high-performance (108 FPS) and a low average trajectory error at the cost of slightly fewer hardware resources compared with some SOTA works.