Motion Robust High-Speed Light-Weighted Object Detection With Event Camera

The event camera asynchronously produces the event stream with a high temporal resolution, discarding redundant visual information and bringing new possibilities for moving object detection. Nevertheless, the existing object detectors cannot make the most of the spatial-temporal asynchronous nature and high temporal resolution of the event stream. For one thing, existing methods fail to consider objects with different velocities relative to the event camera’s motion, resulting from the global synchronized time window with the whole spatial slice. For another, most of the existing methods rely on heavy models and boost the detection performance with low frame rates, failing to utilize the high temporal resolution characteristic of the event stream. In this work, we propose a motion robust and high-speed detection pipeline which better leverages the event data. First, we design an event stream representation called temporal active focus (TAF), which efficiently utilizes the spatial-temporal asynchronous event stream, constructing event tensors robust to object motions. Then, we propose a module called the bifurcated folding module (BFM), which encodes the rich temporal information in the TAF tensor at the input layer of the detector. Following this, we design a high-speed lightweight detector called agile event detector (AED) plus a simple but effective data augmentation method, to enhance the detection accuracy and reduce the model’s parameter. Experiments on two typical real-scene event camera object detection datasets show that our method is competitive in terms of accuracy, efficiency, and the number of parameters. By classifying objects into multiple motion levels based on the optical flow density metric, we further illustrated the robustness of our method for objects with different velocities relative to the camera. The codes and trained models are available at https://github.com/HarmoniaLeo/FRLW-EvD.