DSCS: A Decoupled Semantic Communication System for Video Conferencing

Semantic communication systems utilize neural network models to extract and restore semantic features. However, the extracted semantic vectors often lack comprehensibility for humans. In the context of video conference communication scenarios, we propose a decoupled semantic communication system that takes into account the invariance of human identity features and the variability of mouth shape and posture. The semantic communication system decouples semantic vectors into identity information, mouth shape information, and head pose information. Thus, we only require transmitting the identity feature information of one frame, along with the mouth shape and head pose feature information of each frame. The receiver retains the identity feature information and synthesizes it with the transmitted mouth shape and head pose information for each frame. The method significantly reduces the required transmission bandwidth. Our experiments demonstrate that our video transmission outperforms the baseline methods used for comparison in this paper, particularly at low signal-to-noise ratios (SNR). Furthermore, we separate the semantic vector into interpretable components, empowering users to modify identity features, mouth shapes, and poses, thereby facilitating video re-editing. Significantly, it enhances the flexibility and usability of video editing.