Semantic Communication Based Video Coding Using Temporal Prediction of Deep Neural Network Parameters

Video coding is a critical capability that underpins gaming, entertainment and media ecosystems, enabling effective use of video content in both conventional and non-conventional formats. Semantic communications, where semantics alone can be used to reconstruct media content provided that the context of semantic extraction is known, can effectively implement video coding, but techniques to exploit temporal correlations between video frames to achieve better rate distortion performance with them are just beginning to evolve. A novel approach for this problem of predicting the semantic decoder parameters using temporal correlation is proposed and tested using an autoencoder-based semantic communication system, and the performance is compared with the Neural Network Encoder-Decoder (NNCodec). Experimental results show that it achieves significantly better rate distortion performance compared to NNCodec alone, with PSNR gains between 3 and 25 $d$B depending on the complexity of the video and an average bitrate saving of 54%.