Fast Incremental Least Square Pose Estimation for Hardware Implementation with Rolling Shutter Camera

6 DoF position and orientation estimation is vital for many real-time applications. It is essential to meet low latency, high accuracy and reliability requirements, especially when used for real-time haptic feedback calculation in medical applications. Considering the low latency requirement, designing dedicated hardware promises significant improvements relative to running software on the processor. In our jaw surgery application, we aim to calculate the pose of the rigid object with the tracked marker from the rolling shutter camera by exploiting parallelism and pipe-lining concepts under 1 millisecond latency without sacrificing accuracy on field programmable gate arrays (FPGA) to flow from the camera with rolling shutter effect. Therefore, the iterative least squared algorithm, which is used with high precision in the current applications, has been given an incremental shape to react within the same frame according to the instantaneous data. The algorithm was evaluated by testing with different parameters over the software with synthetically produced fixed and rolling shutter effect images.