SemAudio: Semantic Communication for Audio Streaming Transmission

Semantic communication (SemCom) has been regarded as a promising candidate technology for future intelligent wireless networks by transmitting semantic features. Meanwhile, the rapidly increasing real-time applications bring high demands for audio streaming data processing and exchange. Inspired by this, we propose a SemCom system for audio streaming transmission, named SemAudio. To effectively extract the semantic features of streaming audio, we develop a streaming enhanced Transformer (SET) network to serve as the streaming semantic codec. Specifically, the flexible mask attention network and enhanced memory slot are incorporated in SET to improve the streaming audio reconstruction quality while mitigating the computational complexity. Then, to alleviate the channel noise, we jointly design the semantic and channel coding by integrating the knowledge in both the time and frequency domains. Extensive experimental results demonstrate that the proposed SemAudio outperforms conventional methods in different audio types. Our scheme achieves a better quality-latency tradeoff than traditional communications. Moreover, due to the proposed mask attention strategy, SemAudio can flexibility adjust the latency and adapt to the non-streaming mode.