Development of a hybrid mental speller combining EEG-based brain-computer interface and webcam-based eye-tracking

The main goal of this study was to develop a hybrid mental spelling system combining a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) technology and a webcam-based eye-tracker, which utilizes information from the brain electrical activity and eye gaze direction at the same time. In the hybrid mental spelling system, a character decoded using SSVEP was not typed if the position of the selected character was not matched with the eye direction information (`left' or `right') obtained from the eye-tracker. Thus, the users did not need to correct a misspelled character using a `BACKSPACE' key. To verify the feasibility of the developed hybrid mental spelling system, we conducted online experiments with ten healthy participants. Each participant was asked to type 15 English words consisting of 68 characters. As a result, 16.6 typing errors could be prevented on average, demonstrating that the implemented hybrid mental spelling system could enhance the practicality of our mental spelling system.