Masked Frequency Modeling for Improving Packet Loss Concealment in Speech Transmission Systems

Packet loss concealment (PLC) is crucial for enhancing the quality and intelligibility of speech processing over networks by ensuring the accurate transmission of data, even in the presence of packet loss. In recent years, significant advancements have been made in deep neural network approaches for PLC systems, contributing to substantial improvements in the field. However, despite these advancements, PLC systems have been biased toward reconstructing lost packets, leading to overestimation problems. In this study, we propose a novel approach for training PLC systems using masked frequency modeling as a pre-training method to reduce the artifacts generated by overestimation. In addition, we apply a feature-wise linear modulation layer to the PLC model to capture more fine-grained features by combining previous output features with the reconstructed features. The experimental results demonstrate that the proposed approach outperforms the baseline PLC method in terms of both objective and subjective quality metrics, including PLCMOS, PESQ, STOI, LSD, and WER, thus providing better quality and intelligibility for speech transmission systems. This study presents a new direction for modeling frequency in the PLC algorithm and indicates its potential for practical applications in real-world scenarios.