An Enhanced Hybrid Human-Computer Interaction Strategy Based on Online Classification of Motor Imagery and Eye Tracking

As China’s manned space programs develop, space tasks such as online maintenance, assembly, and adjustment have become increasingly complex and challenging. Astronauts need to interact with the operation interface via portable wearable devices and cooperate with space manipulators to complete the operation tasks. A hybrid human-computer interaction (HCI) system is therefore proposed here based on electroencephalogram (EEG) and eye movement signals, which resolves the "King Midas problem" in the single eye movement interaction mode and facilitate the space manipulator to fulfill tasks. The proposed strategy uses eye movement fixation and instance segmentation to recognize and select operation objects, and employs online decision-making based on motor imagery EEG signals to send interaction instructions, match with task library, and start execution as per the predefined trajectories and tasks. The user selects the region of interest and objects in the interface through eyeball fixation behavior, collects EEG signals synchronously, and makes online classification decision based on characteristic Gaussian enhancement and adaptive dynamic window length (CGDW). Compared with the previous front-end replication with dynamic time window (FRDW) method, our new method has an improved information transfer rate (ITR) and can enable better interaction. In the new method, visual feedback and manipulator state parameter is provided through real-time 3D simulation from the slave, thus avoiding simple feedback loops. Experiments show that the new strategy significantly has significantly reduced the overall interaction time, and surveys demonstrate that the new method achieves higher physical comfort, which verifies the high efficiency of the proposed method.