Task-oriented Approach to Guide Visually Impaired People During Smart Device Usage

Visually impaired persons often experience limited user accessibility regarding smart devices. Particularly, it is challenging for such persons to precisely touch a button or icon to operate smart devices. Existing studies have reported a lack of step-by-step guidance for visually impaired users to perform tasks on smart devices. Although prototypes and non-visual interface systems were implemented to aid blind people in using mobile phones, a step-by-step guidance system was not developed. Moreover, devices other than smartphones were not emphasized. To address these limitations, we propose a novel guidance system that employs an object detection method for recognizing the control panels of smart devices and offers appropriate guidelines based on the user's targeted task. A user study conducted with 20 participants indicated that users were more satisfied with our guidance system than with existing voice assistants.