Content-Aware Modeling and Enhancing User Experience in Cloud Mobile Rendering and Streaming

Cloud mobile rendering (CMR), where compute intensive rendering is performed on cloud servers instead of on mobile devices, can be a promising approach to enable rich rendering based multimedia applications on battery and CPU constrained mobile devices. However, since the video rendered in the cloud has to be streamed to the mobile device over a wireless network with fluctuating and constrained bandwidth, the resulting user experience can be impacted. In the work of Wang and Dey, 2013, adaptive rendering was proposed as a solution, wherein multiple rendering factors can be adapted such that the encoding bit rate of the rendered video is compatible with network bandwidth. However, changing the rendering factors may itself have adverse impact on user experience, which has not been studied earlier. Moreover, the impact can vary depending on the content scene and complexity. In this paper, we analyze the impairment of rendering factors on user experience while considering content characteristics, and combine the rendering impairments with impairments due to video encoding factors (like bit rate and frame rate) and network factors (like bandwidth and delay) to propose a content-aware model to measure user experience. We demonstrate the accuracy of the content-aware model through subjective testing. Furthermore, we use the model to: 1) demonstrate and investigate the tradeoff between rendering quality and encoding quality of the resulting video; and 2) derive a content-aware adaptive rendering (CAAR) algorithm which can dynamically adjust the rendering factors depending on current network conditions, so as to obtain an optimal tradeoff between rendering and encoding quality and maximize the overall user experience at any time. By conducting experiments in a real cellular network, we demonstrate that the proposed CAAR algorithm can greatly enhance the user experience of cloud rendered and streamed mobile video.