A Wearable Device for Indoor Imminent Danger Detection and Avoidance With Region-Based Ground Segmentation

Avoiding objects independently in indoor environments for individuals with severe visual impairment is one of the significant challenges in daily life. This paper presents a wearable application to help visually impaired people quickly build situational awareness and traverse safely. The system utilizes Red, Green, Blue, and Depth (RGB-D) camera and an Inertial Measurement Unit (IMU) to detect objects and the collision-free path in real-time. A region proposal module is presented to decide where to identify the ground from 3D point clouds. The segmented ground area can act as the traversable path, and its corresponding region in the image is removed to prevent detecting painted objects. The system can provide information about the category, distance, and direction of the detected objects by fusing the depth image and the neural network results. A 3D acoustic feedback mechanism is designed to improve the situational awareness for visually impaired people, and guild them traverse safely. The advantage of this system is that our 3D region proposal module can robustly propose the potential ground region and greatly reduce the computation cost of the ground segmentation. Besides, a typical machine-learning-based approach may miss objects because they could not be recognized, though they still may pose a danger. Another advantage of our approach is that the imminent danger detector can detect such unrecognizable objects to help users avoid a collision. Finally, experimental results demonstrate that the proposed system can be a useful indoor assistant tool to help blind individuals with collision avoidance and wayfinding.