Fail-Safe Controller for Personal Mobility Vehicles with BMI

As a hands-free device for personal mobility vehicles, PMVs, we have thus far presented a brain-machine-interface, BMI. The BMI, however, occasionally classified electroencephalography, EEG, signals into wrong control commands. As a result, a PMV exhibited inappropriate behavior and was in danger of collisions with curbs. In order for the PMV to avoid the collisions, we propose a fail-safe controller. For control commands from the BMI, the fail-safe controller determines the final control commands in consideration of surroundings. For this purpose, a depth map that estimates the depth values of an input image is simultaneously generated. The fail-safe controller is based on deep neural networks, such as convolutional autoencoder, CAE, and fully-connected layers. The CAE is used to derive depth maps, and final control commands are determined from feature maps extracted by the CAE through the fully-connected layers. For these multiple outputs, i.e., depth maps and final control commands, multi-task learning is applied to the fail-safe controller. The CAE is trained through deep convolutional generative adversarial network, DCGAN. The fully-connected layers are trained through supervised learning. Furthermore, we introduce undesired control commands that cause collisions into a dataset, especially when the PMV is close to curbs. Finally, we show the effectiveness of the proposed fail-safe controller for collision avoidance.