Design of the Bundle Adjustment FPGA-SoC Architecture for Real Time Vision Based SLAM in AR Glasses

As a method to reduce the load on the CPU in AR glasses, where available resources are limited, research is being actively conducted on a method of additionally attaching a dedicated processor for a specific function. In order to display the augmented image on the augmented reality glasses, it is essential to understand the user's location and surrounding environment, which is mainly calculated using the camera image. Image processing is the process that places the greatest burden on the CPU due to the large amount of data, and the Bundle Adjustment operation, which adjusts and updates the map, occupies the largest portion. Since it is difficult to perform in real time in an embedded environment, various methods have been tried. In this paper, we propose an FPGA-SoC structure that reduces CPU load by designing bundle adjustment operation as a HW module. The Bundle Adjustment module implemented in HW is implemented as a pipeline and parallel processing operation, and in addition, a memory structure is proposed for efficient data input and output of the module. The design is planned to be made with ASIC, and aims to be used as the co-processor of AR glasses.