From RGB-D to low-resolution tactile: Smart sampling and early testing

Because of low resolution and simple scene transducing approaches (e.g., uniform sampling) used in current visual prosthetic devices, only very limited restoration or substitution of vision can be provided to the visually impaired. Two interesting and challenging research problems arise. The first one is how to transduce the most important information to end users with very limited resolution; and the second one is how effectively visual prosthetic devices work with blind people. In this paper, various smart sampling and enhancement methods based on color and depth (RGB-D) segmentation are described that can be used to automatically select, sample and transduce the most useful scene information to end users with these visual substitution devices. The RGB-D segmentation is first generated by a patch-based stereovision algorithm, given a pair of color images captured by a stereo camera head. Then, the RGB-D segmentation is transduced into various alternative perception choices. Furthermore, some early studies on a visual prosthetic device using a tongue stimulator are described, and preliminary experiments are discussed that test the proposed methods.