EESC-S: An Emotion-Enhanced Semantic Communication Framework for Speech Transmission

The deep learning (DL) enabled semantic communication technology has been revealed its capability in improving the transmission efficiency by recent fruitful research. Speech transmission is a vital application for semantic communication, and the emotion contained in a speech are important in accurately understanding the speaker (i.e., transmitter). This paper proposes an emotional information assisted semantic communication framework for speech transmission. In this framework, the content semantic and the emotional semantic of speech are jointly extracted to achieve semantic consistency between the transmitter and the receiver. Specifically, the emotion-semantic fusion mechanisms are designed to enhance the correlation between the content and the emotion at the receiver. The speech emotion recognition (SER) task is also considered and the intermediate feature of classification is used to construct emotion perception loss, ensuring perceptual similarity between the reconstructed speech and expected emotion category at the utterance level. Extensive simulation results demonstrate that the proposed framework can significantly improve the speech transmission performance.