A Lightweight Multi-Feature Fusion Attention Network for Single-Channel Speech Separation

Speech separation is a fundamental task in speech signal processing and plays a critical role in applications such as automatic speech recognition. Although deep learning has led to significant progress in single-channel speech separation, challenges persist in feature extraction, phase reconstruction, and computational efficiency. In this paper, we propose a lightweight multi-feature fusion attention convolutional network (LMFA-Net) for end-to-end single-channel speech separation. The encoder integrates a deterministic Multi-Phase Gammatone Filterbank (MP-GTF) with a Convolutional Efficient Channel Attention (ECA) module to enhance time-frequency feature representation. The separation module employs Ghost Module-based lightweight convolutional blocks to reduce model size and complexity while maintaining strong separation performance. In the decoder, outputs from transposed convolution and the pseudo-inverse MPGTF are fused via an attention-based weighting mechanism to reconstruct waveforms. Experimental results on WSJ0-2mix, Libri2Mix, and a self-built Chinese dataset, NJUST603-2Mix, show that LMFA-Net achieves superior SDRi and SI-SNRi scores with fewer parameters, demonstrating its effectiveness and strong generalization capability.