Smart sampling and transducing 3D scenes for the visually impaired

Current visual prosthetic devices provide only very limited restoration or substitution of vision for the visually impaired, in part due to their low resolution and simple scene transducing approaches with uniform sampling and quantization. In this paper, a novel smart sampling method using a color-patch-based stereo reconstruction approach is described to automatically select, sample and transduce the most useful scene information to end users using visual substitution devices. The proposed method first constructs a patch-based 3D model of the scene using the color-patch-based stereovision algorithm given a pair of video frames captured by a stereo camera head. Then, the patch-based 3D model is analyzed using the smart sampling algorithm and further transduced into various alternative perception choices, using both color and depth information. Some preliminary experimental results are shown to validate the proposed method.