TIME-E2V: Overcoming limitations of E2VID

In the field of action recognition, event cameras have marked a breakthrough by capturing motion dynamics beyond the capability of traditional cameras, thanks to their high temporal sensitivity. However, the asynchronous and sparse nature of event data challenges their use with traditional convolutional neural networks (CNNs). The E2VID model offers a solution by transforming event data into continuous video frames, enabling the use of standard CNNs for event-based data analysis. However, it struggles with accurately capturing motion speed variations and pauses, limiting its effectiveness in scenarios where temporal dynamics are crucial. In response, we introduce TIME-E2V, which integrates spatial embeddings from E2VID with LSTM-derived temporal embeddings from frame timestamps. This combination is processed by a modified 3D convolutional network (C3D), leveraging its inherent strengths in video analysis. Our proposed approach not only overcomes E2VID’s challenges but also delivers competitive performance across a wide range of dynamic scenes with the leading action recognition networks for event cameras, including those based on Spiking Neural Networks.