Pose Estimation for Event Camera Using Charuco Board Based on Image Reconstruction

Event cameras offer attractive properties compared to conventional frame-based cameras, such as high temporal resolution, very high dynamic range, and low power consumption. Thanks to these characteristics, event cameras have a great potential for sensing challenging lighting or high motion conditions in computer vision tasks and robotics applications. Traditional patterns such as chessboard and circle grid based methods have been proposed to calibrate or estimate the pose of the event camera. However, these methods are less versatile as they require the entire board to be visible in all images and do not allow occlusion. To overcome these limitations, this paper proposes a new method to estimate the 6DoF pose of an event camera using a Charuco board based on image reconstruction with a deep learning approach. Using images reconstructed from the event streams captured of the Charuco board, it can be successfully estimated the 6DoF pose of the event camera even in the presence of occlusion. Experiments performed in a simulation environment show the effectiveness of the proposed method.

I. Introduction

In recent years, the demand for 3D sensing technology has increased. Conventional frame-based cameras have typically been used to capture information. A conventional frame-based camera captures a scene by accumulating photons reflected from objects over a period of time to produce an image. Conventional cameras have been used in many studies on 3D sensing due to their advantage of high-resolution images. However, conventional frame-based cameras often suffer from low frame rates, high latency, or poor adjustment to extreme lighting conditions.