Enhancing Video Encoding for Cloud Gaming Using Rendering Information

Cloud gaming allows games to be rendered on the cloud server and allows the rendered videos to be encoded and streamed in real time to the player's devices. Compared with other video streaming applications, cloud gaming offers a unique opportunity to enhance the video encoding process by exploiting rendering information. In this paper, we propose two techniques to improve cloud gaming video encoding, aiming at enhancing the perceived video quality and reducing the computational complexity, respectively. First, we develop a rendering-based prioritized encoding technique to improve the perceived game video quality according to network bandwidth constraints. We first propose a technique to generate a macroblock (MB)-level saliency map for every game video frame using rendering information. Furthermore, based on such a saliency map, a prioritized rate allocation scheme is proposed to dynamically adjust the value of quantization parameter of each MB. The experimental results indicate that the perceptual quality can be greatly improved using the proposed technique. We also develop a rendering-based encoding acceleration technique that utilizes rendering information to reduce the computational complexity of video encoding. This technique mainly consists of two parts. First, we propose a method to directly calculate the motion vectors (MVs) without employing the compute intensive motion search procedure. Second, based on the computed MVs, we propose a fast mode selection algorithm to reduce the number of candidate modes of each MB. The experimental results show that the proposed technique can achieve more than 42% saving in encoding time with very limited degradation in video quality.