Frame rate reduction of depth cameras by RGB-based depth prediction

Depth cameras are becoming widely used for facilitating fast and robust natural user interaction. But measuring depth can be high in power consumption mainly due to the active infrared illumination involved in the acquisition process, for both structured-light and time-of-flight technologies. It becomes a critical issue when the sensors are mounted on hand-held (mobile) devices, where power usage is of the essence. A method is proposed to reduce the depth acquisition frame rate, possibly by factors of 2 or 3, thus saving considerable power. The compensation is done by calculating reliable depth estimations using a coupled color (RGB) camera working at full frame rate. These predictions, which are shown to perform outstandingly, create for the end user or application the perception of a depth sensor working at full frame rate. Quality measures based on skeleton extraction and depth inaccuracy are used to calculate the deviation from the ground truth.