A Human-Vehicle Collaborative Simulated Driving System Based on Hybrid Brain–Computer Interfaces and Computer Vision

Automatic driving vehicles have been developed to provide more convenient and comfortable driving experiences. However, these vehicles failed in satisfying the variance of human intentions. Recently, the strategy of collaborating brain-computer interface (BCI) controlling and automatic driving receives attention. Since the BCI system remained some limitation in real-time controlling, a fusion method has been proposed to explore and verify the feasibility of human-vehicle collaborative driving in this paper. A hybrid BCI was developed to interpret human intentions. In addition, a computer vision-based automatic driving component was developed to maintain the vehicle on the road. A system for fusing these two kinds of vehicle driving decisions was first proposed in this paper. This system can simultaneously obtain the visual data and the hybrid electroencephalograph (EEG) signals. The hybrid EEG signals consist of steady-state visual evoked potentials and motor imagery. The obtained multisource information can be fused to make the final decision to drive a simulated vehicle. The proposed system was evaluated with different destinations. The experimental results verify the feasibility of fusing both human intention and computer vision. The task success rate reached 91.1% and the information transfer rate was 85.80 bit/min.