Optimal Transport With Mamba for Multimodal Inertial Signal Enhancement

As motion-sensing components, multimodal inertial sensors composed of accelerometers and gyroscopes are recognized for their compact size, low cost, and broad applications in wearable and smart devices, but they are affected by severe noise. Wavelet transform is renowned for its flexibility in analyzing signals due to its diverse wavelet bases, allowing it to adapt to different signal characteristics. However, the diverse signal noises challenge wavelet allocation. Moreover, the modal differences between acceleration and gyroscope signals make it difficult to share the same wavelet, adding complexity to the wavelet allocation for multimodal signals. To this end, we propose an OT-Mamba framework, which leverages Mamba to extract signal features. Mamba is specifically designed to handle ultra-long sequences, allowing it to capture long-range temporal dependencies for better wavelet selection. Considering the heterogeneity and potential synergy between the accelerometer and gyroscope signals, an optimal transport interaction is proposed to mine their relationship for collaborative wavelet selection. The proposed OT-Mamba combines the reliability of wavelet-based methods and the flexibility of deep learning approaches. As a weakly supervised method, OT-Mamba achieves superior performance compared to existing methods (including fully supervised ones) and outperforms the current state-of-the-art method by an order of magnitude across all quantitative metrics and downstream tasks.