Sketch Graph Representation for Multimedia Computational Communications: A Learning-Based Method

Multimedia computational communications towards 6G can improve the transmission efficiency significantly by introducing intelligent computation in the communication process. This intelligent/smart communication architecture includes multimedia representation, coding, transmission and other parts from the perspective of semantics, where multimedia semantic representation is the core part and is mainly utilized to reduce the amount of multimedia data. In this paper, sketch graph is proposed as an effective representation of images to describe the pixel variations, geometric feature distribution and structural information and has potential applications in multimedia computational communications. Specifically, we developed a learning-based method to extract sketch graphs with edge detection, sketch point detection and sketch line detection by deep neural networks (DNNs). Moreover, we designed an end-to-end extraction method and achieved real-time processing. The experimental results on several datasets demonstrated the advanced performance in terms of classification and generation tasks. Image classification results on the HumanSketch, ImageNet, and Caltech datasets showed that sketch graphs extracted by our method had better describing ability than those extracted by traditional methods and other traditional image compression methods. On the other hand, image generation results on the Cityscapes dataset indicated the potential of the sketch-graph-based image compression codec.