MDAF: Enhancing Temporal Dynamics in CNNs for Action Recognition

Understanding temporal information in video sequences is crucial for various computer vision tasks, such as action recognition. Transformer-based methods and GCNs can effectively handle temporal information, but they suffer from high computational costs or poor generalization capabilities. Some research has shown promising results using 3D CNNs to process skeleton data instead of RGB data, but 3D CNNs may treat temporal dimension as a spatial one, limiting their ability to capture complex temporal dynamics. To address these issues, we propose the Multi-Dimensional Attention Fusion module (MDAF), which includes separate channel, spatial, and temporal attention computation modules, integrated through an attention fusion module. This design ensures the effective utilization of attention information across all dimensions. Our method enhances temporal dynamics, improving the recognition of complex actions. Experiments conducted on four benchmark datasets demonstrate that our approach outperforms state-of-the-art methods, achieving significant performance improvements.