Improved Event-Based Dense Depth Estimation via Optical Flow Compensation

Event cameras have the potential to overcome the limitations of classical computer vision in real-world applications. Depth estimation is a crucial step for high-level robotics tasks and has attracted much attention from the community. In this paper, we propose an event-based dense depth estimation architecture, Mixed-EF2DNet, which firstly predicts inter-grid optical flow to compensate for lost temporal information, and then estimates multiple contextual depth maps that are fused to generate a robust depth estimation map. To supervise the network training, we further design a smoothing loss function used to smooth local depth estimates and facilitate estimating reasonable depth for pixels without events. In addition, we introduce SE-resblocks in the depth network to enhance the network representation by selecting feature channels. Experimental evaluations on both real-world and synthetic datasets show that our method performs better in terms of accuracy when compared to state-of-the-art algorithms, especially in scene detail estimation. Besides, our method demonstrates excellent generalization in cross-dataset tasks.

I. Introduction

Bio-inspired dynamic vision sensors called event cameras, such as Dynamic Vision Sensor (DVS) [1], are novel eventdriven devices that only report the pixel where illumination intensity has changed beyond a set threshold. Unlike conventional sensors that capture a global intensity image, as shown in Fig. 1, event cameras output asynchronous event streams, which consist of positive and negative events that indicate the increase or decrease in illumination, respectively. Each event contains the timestamp it occurs, the pixel-location, and polarity information of brightness changes. With the advantages of high temporal resolution, high dynamic range, low power consumption, and no motion blur, event cameras offer a potential choice for the scenarios where conventional cameras are challenged [2], [3].