Sensor-based real-time adaptation of 3D video encoding quality for remote control applications

The availability of stereoscopic mobile devices, such as mobile phones, on the consumer market allows to attempt the development of low-cost remote control systems that can provide a real-time 3D video feedback. In this work we show how implement such a communication system by considering the stringent latency constraints of the remote control scenario. To reduce the impact of this issue, we observe that part of the latency is due to the limited processing power of the mobile device that cannot sustain video transmission at high quality with low latency. Thus, we propose to dynamically change the latency-quality trade-off at the transmitter to optimize the quality of experience as perceived by the operator of the remote control system, by taking into account, in real-time, the dynamics of the control operations. In more details, low-cost accelerometer and gyroscopic sensors are employed to decide in real-time how much latency has to be privileged over quality and vice versa, by selectively reducing the quality of one of the views in favor of a reduced overall latency. Comparisons with a non-adaptive higher-quality but also higher-latency system show that the operators prefer the adaptive system despite the video quality is slightly reduced in dynamic control conditions.