Multidimensional Cockpit Perception via Mutual Information-Guided Signal Graph Fusion

Multidimensional Cockpit Perception (MCP) is an indispensable technology in assisted driving systems for modern vehicles, which requires joint recognition of the driver’s emotional behaviors, traffic context, and vehicle condition. Despite promising progress in 2D to 3D neural networks, semantic discrepancies and intricate temporal dependencies among different signal streams still cause serious performance bottlenecks. To address these challenges, this paper proposes a Mutual Information-guided signal Graph fusion (MIG) framework for robust cockpit perception. MIG introduces an adaptive mutual information constraint to maximize the mutual information among different perceived signal streams, thus reinforcing the task-related feature semantic consistency. In addition, a signal graph fusion module is designed to capture intra- and inter-signal elemental correlations at a fine-grained level to learn joint multi-signal representations used for different downstream tasks. Systematic experiments on the MCP benchmark verify the effectiveness of the proposed framework and the necessity of our components.