Mamba-Based Temporal Modeling for Event-Based Lip Reading

Event-based lip reading is gaining attention for its potential to capture high-speed lip movements with low latency and power consumption. However, effectively modeling the sparse and asynchronous nature of event streams remains a key challenge. We propose E-Mamba, a novel temporal modeling framework that alternately stacks Mamba blocks and GRU modules to jointly capture long-range dependencies and local temporal dynamics. The Mamba block offers efficient global context modeling via linear state-space representations, while the GRU enhances nonlinear modeling capability and local sensitivity through gated mechanisms. Experiments on the DVS-Lip dataset demonstrate that E-Mamba achieves 73.23 % accuracy, outperforming existing event-based lip reading methods. Ablation studies further confirm the complementary strengths of the two components, highlighting the effectiveness of our design.