Enhancing Multimodal Action Recognition With Semantic-Guided Feature Disentangled and Multichannel Adaptive Feature Fusion

Multimodal action recognition exploits the consistency and complementarity among different modalities to mitigate measurement biases inherent in single-modality data. However, existing methods primarily focus on intermodal correlation, often at the expense of modality-specific semantic features. This limits them to discerning ambiguous actions when different actions with similar sequences, as crucial modality-specific nuances are lost in the fused representation. To address the aforementioned issue, we propose feature disentangled representation learning for multimodal action recognition (FDRL-MAR). Unlike previous approaches that blend multimodal information indiscriminately, FDRL-MAR explicitly disentangles action features into modality-common and modality-specific representations. This ensures that crucial modality-specific cues, such as skeleton posture details or inertial motion signatures, remain distinguishable while preserving cross-modal consistency. Furthermore, since action recognition is inherently dynamic, we introduce a multichannel adaptive fusion mechanism that dynamically adjusts the contribution of different semantic features. Finally, extensive experiments on three benchmark datasets demonstrate the effectiveness of FDRL-MAR over state-of-the-art methods. Our code and data will be released on GitHub: https://github.com/YooLiuXiao/FDRL-MHAD