A Novel Hybrid Brain–Computer Interface Combining the Illusion-Induced VEP and SSVEP

Traditional single-modality brain-computer interface (BCI) systems are limited by their reliance on a single characteristic of brain signals. To address this issue, incorporating multiple features from EEG signals can provide robust information to enhance BCI performance. In this study, we designed and implemented a novel hybrid paradigm that combined illusion-induced visual evoked potential (IVEP) and steady-state visual evoked potential (SSVEP) with the aim of leveraging their features simultaneously to improve system efficiency. The proposed paradigm was validated through two experimental studies, which encompassed feature analysis of IVEP with a static paradigm, and performance evaluation of hybrid paradigm in comparison with the conventional SSVEP paradigm. The characteristic analysis yielded significant differences in response waveforms among different motion illusions. The performance evaluation of the hybrid BCI demonstrates the advantage of integrating illusory stimuli into the SSVEP paradigm. This integration effectively enhanced the spatio-temporal features of EEG signals, resulting in higher classification accuracy and information transfer rate (ITR) within a short time window when compared to traditional SSVEP-BCI in four-command task. Furthermore, the questionnaire results of subjective estimation revealed that proposed hybrid BCI offers less eye fatigue, and potentially higher levels of concentration, physical condition, and mental condition for users. This work first introduced the IVEP signals in hybrid BCI system that could enhance performance efficiently, which is promising to fulfill the requirements for efficiency in practical BCI control systems.


I. INTRODUCTION
B RAIN-COMPUTER interface (BCI) is a technology that enables individuals to control external devices by translating brain signals into commands, without the need for peripheral nerves or muscles [1].Electroencephalography (EEG) has been extensively used for measuring brain activity in the field of BCI research because of its non-invasive, easily portable, and low-cost properties [2].BCI systems utilize diverse EEG patterns for the purposes of device control, communication, and neural rehabilitation [3], [4], [5].In EEGbased BCI studies, there are three classic modalities of BCI systems, including event-related potential (ERP), steady-state visual evoked potentials (SSVEP), and motor imagery (MI).
Stimulus-elicited ERP responses can be extracted as brain control signals for BCI systems, such as the P300, motiononset visual evoked potential (mVEP), N170, vertex positive potential (VPP), and illusion-induced visual evoked potential (IVEP).Among them, the P300 component has been widely employed for BCI systems within the framework of oddball paradigm [6].P300 response is evoked approximately 300 ms after the presence of an intended stimulus, characterized by its low probability of occurrence.To identify P300 from background noise, multiple EEG epochs are averaged to enhance signal-to-noise ratio (SNR).However, it may increase the stimulus duration and reduce the communication rate [7].The mVEP mainly refers to the N200 component, which is the neural response with a negative deflection around 200ms after the onset of actual motion [8].Compared to P300 BCI system, it is more comfortable for users, but it also has losses in BCI speed due to the requirement for averaging.The N170 and VPP have been validated to be face-sensitive ERP responses associated with the neural processing of face [9].Many P300-based BCI studies utilize facial images to replace flashing characters for stimulus intensification in P300 speller paradigm and obtained better performance when compared to the conventional P300 BCI [10], [11].Recently, the IVEP has been introduced to BCI systems and evidenced the stable modulation of ERP components N1 and P2 by illusory stimuli in our previous study [12].
The SSVEP-based BCI has caught increasing attention from researchers over the past few decades because of its high information rate, reliability, and no training requirement [13], [14], [15].In the SSVEP paradigm, all stimuli are presented simultaneously and each flickering at a specific frequency independently.When the user focuses on one of the flickering stimuli, robust EEG responses with corresponding frequency and harmonics are evoked.Among EEG-based BCIs, SSVEP is considered to be the fastest BCI approach.However, longterm exposure to repetitive flashing stimuli may cause user fatigue or even epileptic seizures.
The MI approach applies the synchronization and desynchronization (ERD/ERS) of sensorimotor rhythm to identify control command [16], [17].In MI paradigm, generating neural response requires subjects to perform endogenous mental tasks, for instance imagining hand movements, which necessitates sufficient training to acquire recognizable features [18].Compared to exogenous BCI systems which depend on external stimuli such as ERP and SSVEP, the implementation of MI BCI is more restricted by BCI illiteracy among users [19].
For BCI communication and control, the attempt to improve BCI performance has always concentrated on speed and accuracy [1].In addition to signal analysis techniques and classification algorithms that have been well-studied, expanding the dimension of features is also a promising approach to improve BCI performance.In this regard, the development of hybrid BCI has emerged as an effective direction, which combines two or more brain patterns.Here we are interested in the EEG-based hybrid BCI systems, which are considered pure hybrid BCIs since they do not require additional signal acquisition equipment [20].Recently, many efforts have been made to combine two types of EEG signals in hybrid BCI systems for communication that can improve information transfer rate (ITR), accuracy, and usability.For example, Combaz and Van Hulle proposed a hybrid visual P300-SSVEP paradigm for icons recognition, which leads to a significant improvement in ITR compared to conventional P300 and SSVEP [21].Allison et al. presented a hybrid BCI that can simultaneously detect ERD and SSVEP activity for two-dimensional control by imagining movement while focusing on flickering targets [22].Although the accuracy was increased, most subjects were still unable to communicate effectively due to the task complexity.Indeed, many studies have endeavored to evoke SSVEP and other ERP signals simultaneously in order to enhance the efficiency of BCI control or communication, as demonstrated in various applications such as BCI spellers and cursor control systems [23], [24].Both P300 and SSVEP are exogenous EEG responses elicited by external stimuli, which suffer little from BCI illiteracy and are easy to extract reliable features.These hybrid systems leverage multiple characteristics to enhance the distinguishability of different stimuli and expand the accessibility of BCI systems for users.In reported synchronous hybrid BCI studies, the adopted ERP signals such as P300 and mVEP utilize a time-division coding scheme where multiple targets are scheduled in different time intervals [25].However, the IVEP method exploits spatio-temporal discrepancy features for classification instead of detecting the specific time slot in which the evoked component appears, which provides room for reducing target detection time.For the goal of enhancing both speed and accuracy, there is great potential to combine IVEP and SSVEP.
In this study, we designed a novel IVEP-SSVEP hybrid paradigm and evaluated performance in a 4-command classification task.To assess its feasibility, two different experiments were conducted in our study.In the first experiment, the static IVEP paradigm was conducted to examine the distinguishability between different motion illusions.Different visual motion illusions were presented simultaneously and evoked IVEP signals (consists of N1 and P2 ERP components) with distinct spatio-temporal characteristics.The second experiment comprised two paradigms: the proposed hybrid paradigm, which integrated SSVEP and IVEP stimuli by periodically superimposing motion illusions on flickering, and the conventional SSVEP paradigm, which served as a control condition for performance comparison.Spatial filters enabled the co-extraction of multiple features that were time-locked to the stimuli from EEG background noise.The results demonstrate that the IVEP-SSVEP hybrid BCI outperforms the SSVEP-based BCIs in terms of classification accuracy and ITR.We also investigated the user experience by employing a questionnaire to evaluate the subjects' conditions.The collected self-assessment data provide evidence that proposed hybrid BCI offers less eye fatigue.Thus, these findings suggest that our hybrid paradigm can enhance BCI performance without imposing additional burdens on users.
The remaining parts of this paper are organized as follows.The experimental setup and methods are introduced for the proposed paradigm in Section II.The results of data analysis and performance are presented in Section III.Discussions regarding the experimental results are presented in Section IV, and the conclusions are given in Section V.

A. Participants and Data Acquisition
Twelve healthy subjects (3 females and 9 males, aged between 22 and 25) from ShanghaiTech University participated in this experimental study.All participants had normal or correct-to-normal vision and did not have any neurological diseases that could have affected the experimental results.The study protocol was approved by the Ethics Committee of ShanghaiTech University, and all subjects provided written informed consent prior to the experiment.During the experiment, subjects followed the procedural instructions.Each participant was paid for their engagement at the end of experiment.
EEG signals were recorded using a Neuroscan SynAmps2 system equipped with a 64-channel electrode cap, where the electrode distribution was consistent with the international 10-20 system.Subjects were seated in front of the monitor with a viewing distance of approximately 70 cm.All experiments were performed in a dark electromagnetic shielded room to eliminate external noise.The impedance of each electrode was kept below 10 k .Both IVEP and SSVEP were elicited by external visual stimuli, which were associated with the visual cortex.In this study, we adopted nine electrodes overlaying parietal and occipital areas including Oz, O1, O2, Pz, POz, PO3, PO4, PO5 and PO6.The electrodes were chosen as their established contribution to the recognition of SSVEP and IVEP signals, as demonstrated in prior studies [12], [26].The reference electrode was placed in the central area and the ground electrode was placed in the frontal area.EEG data were sampled at 1000 Hz and then filtered at 50 Hz with a notch filter to exclude power line interference.

B. Stimulation Design
For this study, three paradigms were involved to assess the feasibility of proposed hybrid BCI paradigm.These paradigms were displayed on an LCD monitor with a resolution of 1920 × 1080 pixels and a refresh rate of 60 Hz.The user interface layout for all three paradigms consisted of four target selection items which were positioned on the left, right, up, and bottom of the monitor, respectively (see Fig. 1).Each stimulus had a fixed size of 250 × 250 pixels, corresponding to a visual angle of 4.5 • horizontally and vertically.The paradigms was implemented by the Psychophysics Toolbox [27] and programmed in MATLAB 2020b.The stimulation designs of three paradigms are as follows.
1) IVEP: For IVEP stimulation, we applied four static motion illusions designed by A. Kitaoka (see Fig. 1(a)) [28].All chosen illusory stimuli incorporated the elements to generate the illusory perception of rotational motion that an object is rotating when no physical rotation is actually occurring.Previous studies have found that the neural activities associated with illusory motion perception and actual motion perception were indistinguishable [29].Moreover, our recent study has observed that the evoked oscillatory responses for different visual illusions exhibit distinct characteristics [12].In this study, we replicated the paradigm by presenting stimuli in a stationary manner to explore the characteristics of evoked responses for illusory stimuli.
2) SSVEP: The conventional SSVEP stimuli were encoded with phase and frequency.Four distinct SSVEP stimuli were generated with each stimulus flickering at frequencies Fig. 2. Illustration of the experimental procedures for Experiment 1 and 2. Experiment 1 consists of 20 blocks for IVEP task and each block includes four trials for four targets.In Experiment 2, there are two SSVEP and hybrid tasks and each task contains 10 blocks.A questionnaire for subjective evaluation is conducted at the end of Experiment 2. of 14 Hz, 13 Hz, 12 Hz, and 11 Hz, respectively.The phase interval between the stimuli was set at 0.5 π (see Fig. 1(b)).
3) Hybrid: The designed hybrid stimuli displayed in this paradigm was a periodic superposition of IVEP stimuli on the SSVEP flickering stimuli.The stimulus settings for flickers and motion illusions were consistent with those employed in the IVEP and SSVEP paradigms.Fig. 1(c) depicts the stimulus modulation for four hybrid stimuli in each frame within one second.Considering that the latency for the evoked N1 and P2 component typically falls within the range of 100-200 ms and 200-300 ms, the stimulus onset asynchrony (SOA) for the illusory stimuli was set to 400 ms to avoid overlaying.Thereby the presentation frequency of all illusory stimuli was 2.5 Hz, with a fixed duty cycle of 12.5% (50 ms the illusory stimuli on and 350 ms off).The illusory stimuli were periodically presented after SSVEP stimuli onset 200 ms, and the stimulation procedure of hybrid paradigm is shown in Fig. 1(d).

C. Experimental Procedures
The experimental studies comprised two distinct steps: Experiment 1 aimed to examine the distinguishability of the IVEP paradigm, while Experiment 2 focused on performance evaluation by comparing the proposed hybrid paradigm with SSVEP paradigm.All experiments were conducted on the same day for each subject to ensure consistent experimental conditions.The whole procedures for Experiment 1 and 2 are illustrated in Fig. 2.
In Experiment 1, the IVEP paradigm was conducted independently.Each subject completed 20 blocks and each block consisted of 4 trials corresponding to the four targets.Thus, there were 20 trials for each stimulus in total and 80 trials for the entire experiment.During each trial, a 2 s visual cue in the form of a red dot instructed subjects to focus on the assigned target.This was followed by a 5 s stimulus period, during which all stimuli were simultaneously presented.The order of attention targets within each block was random.
In Experiment 2, classification tasks were conducted to compare the performance of hybrid paradigm and SSVEP paradigm.A questionnaire for subject self-assessment was also utilized.The classification tasks consisted of two SSVEP tasks and two hybrid tasks, with a total of 40 blocks (10 blocks per task).The procedure design in each block and trial followed the structure implemented in Experiment 1.To mitigate the potential impact of order effects on the experimental results, the presentation sequence of the two tasks was counterbalanced among subjects.
Following the stimuli tasks in Experiment 2, a questionnaire was employed to evaluate subjects' experiences, as it is also considered an important aspect of BCI performance.Four representative subjective states, namely eye fatigue, concentration, physical condition, and mental condition, were selected for assessment [19].Subjects were required to rate each state based on the 5-point Likert scale.For eye fatigue and concentration, the scale ranged from 1 to 5 represented the degree from lowest to highest, while for physical condition and mental condition, it represented the degree from worse to best.

D. Data Processing and Analysis
We collected a total of 240 trials from each subject for the entire experimental study.The raw EEG data were filtered using a band-pass filter ranging from 1 to 30 Hz and segmented from -200 to 800 ms in Experiment 1. Baseline correction was performed using the 200 ms period preceding the stimulus presentation.
In the ERP evoked model, the signals s(t) are typically regarded as being independent of the unrelated ongoing EEG activity which is considered as random noise n(t) [30].To visualize task-related ERP, multiple EEG epochs were averaged across trials and subjects for each time point, then random noise varied from trial to trial was effectively attenuated, hence the ERP measure method is where N is the number of subjects, K is the number of trials, i is the subject index, and j is the trial index.The ERP Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
components were evaluated by measuring their mean amplitudes and local peak latencies within specific time windows.
In Experiment 2 study, data epochs were extracted from 0 to 2 s for grand average waveforms analysis of the EEG responses elicited by hybrid stimuli and SSVEP stimuli.For the analysis in frequency domain, 5 s data epochs were filtered by the bandstop filter ranging from 1 to 10 Hz in order to exclude the fundamental frequency (2.5 Hz) and part of the harmonics caused by the illusion stimulus in hybrid task.In this study, the 140 ms latency delay was considered in epoch extraction for performance evaluation [31].
Given the hybrid features incorporating both IVEP and SSVEP, two different methods are applied to estimate the SNR.For SSVEP responses, we computed SNR by computing the ratio of Fourier power at stimulus frequencies to the mean power in adjacent frequencies [32].Then the SNR of SSVEP responses in decibels (dB) can be estimated as where f denotes the frequency, P(f) represents the power of signal, and the frequency step is set to 0.2 Hz.The n is set to 6 then the adjacent frequency range of ±0.6 Hz.
For waveform estimation, the SNR in time course is computed as the ratio between mean amplitude of EEG responses and baseline noise [33].In specific, we extract the epochs from 200 ms to 600 ms after stimulus onset, which includes the entire stimulus period containing both IVEP and SSVEP responses for hybrid paradigm.Noise is assessed within the baseline interval of −400 ms to 0 ms before the stimulus onset.The SNR of waveform expressed in dB is computed as follows where T represents the number of time points, X s denotes the average feature-relevant waveform, and X n denotes the average baseline waveform.

E. Classification
We employed task-related component analysis (TRCA) algorithm to detect targets, which has been introduced to SSVEP-based BCI and achieved high accuracy and ITR [34].Note that the spatio-temporal characteristics of evoked ERP and SSVEP vary from different stimuli and they are time-locked to stimulus onset.Hence, in the case of hybrid paradigm, the TRCA enables simultaneous extraction of features without additional separate extraction steps for IVEP and SSVEP.Ideally, it is assumed that the task-related component is linearly separable from the background EEG activities in line with the linear model of evoked ERP.The EEG signals for the l-th trial is assumed as X l ∈ R N c ×N s , l = 1, 2, . . .,N t , where N c is the number of channels, N s is the number of sampling points, and N t is the number of trials for each stimulus.The linear combination of l-th trial is formulated as where the spatial filter w ∈ R N c×1 is the goal that TRCA method aims to optimize.
To obtain optimal spatial filter, the covariance of all possible combination of trials needs to be maximized, which is performed as where represents the signals of i-th channel in l 1 -th trial, and the X (l 2 ) j represents the signals of j-th channel in l 2 -th trial.Here denotes a concatenated matrix of all trials H = [X 1 , X 2 . . .X N t ].The variance of Y l normalized to one is calculated as where j 1 and j 2 are the index of channels.Then the constrained optimization is given by the Rayleigh-Ritz problem Thereby, the spatial filter is obtained from the eigenvector corresponding to the maximum eigenvalue.The individual template X is computed by averaging all trials of the training data.Given the optimal spatial filter, the correlation coefficient between the test signals and corresponding individual template of n-th stimulus is calculated as The filter bank analysis was applied to decompose signals into sub-band components in order to effectively extract the harmonics [31].The bandwidth for each subband m-th is defined as [m×8 Hz, 90 Hz], m = 1, 2, . . .,N m , where N m represents the number of subbands.The N m is set to 5, and the low-pass cut-off frequency for the first subband is set to 0.5 Hz to accommodate the frequency range relevant to IVEP signals in hybrid paradigm.Then the targeted stimulus can be identified as where N k is the total number of stimuli, and a(m) = m − p +q.For the traditional SSVEP paradigm, the constants are set to p = 1.25 and q = 0.25, respectively.Considering the potential influence of IVEP on SSVEP, we optimized the parameters utilizing the grid search approach in which p ranges from 0 to 2 and q ranges from 0 to 1, using a step size of 0.1.Following optimization, the parameters p and q for the hybrid paradigm are 0.5 and 0.1.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I LOCAL PEAK LATENCY AND MEAN AMPLITUDE OF THE N1 AND P2 COMPONENTS (MEAN ± SEM)
Fig. 3. Grand average ERP waveforms for IVEP paradigm under four conditions in Experiment 1. ERP responses are averaged over all electrodes, trials, and subjects.The vertical dashed line marks the stimulus onset, and the fields in different colors represent the standard error of mean (± 1 SEM) corresponding to each illusory stimulus.
To evaluate and compare the performance of hybrid and SSVEP paradigms, we also calculated the ITR.The computation of ITR in bits/minute defined by Wolpaw et al. [1] is shown as follows.
where N represents the number of choices, P denotes the classification accuracy, and T is the average time for each selection, consisting of 0.2 s to 2 s of stimulation time (at 0.2 s intervals) and 2 s of gaze shift time.

A. Characteristics Analysis
We first investigated the EEG responses evoked by four different illusory stimuli in Experiment 1. Fig. 3 depicts the grand average waveforms under four conditions.Based on the waveforms, the latency range of the N1 and P2 components of interest were set between 110-210 ms and 210-350 ms, respectively.In this study, we conducted one-way repeated measures analysis of variance (ANOVA) to assess significance of difference.The normality assumption was validated through the Kolmogorov-Smirnov (KS) test.In cases where the assumption of homogeneity of variance was not met, we applied Welch's ANOVA as an alternative method.Here, we abbreviate the stimuli according to their corresponding locations (L, U, R, and B).Within the defined latency range, negative deflection N1 and following positive deflection P2 were shown in the waveforms for L, U, R, and B conditions.With statistical tests, the amplitude of waveforms for N1 and P2 both show significant difference ( p < 0.0001).ERP quantification analysis for N1 and P2 components involved local peak latency and mean amplitude detected from the defined latency range.In detail, mean amplitude was calculated from the mean voltage in fixed latency range, and local peak latency was extracted at the local maximum amplitude.Table I shows the grand average local peak latency and mean amplitude for each static visual illusion condition.The results of statistical comparison are presented as p-values.Across subjects, the local peak latency of N1 component shows statistically significant differences ( p < 0.05), while that of P2 component shows no significant differences ( p = 0.57), and there is also no significant difference in mean amplitude under four conditions (N1: p = 0.67, P2: p = 0.81).The Tukey-Kramer post-hoc test was employed to conduct pairwise comparison of local peak latency of the N1 across four conditions.The results reveal a statistically significant mean difference in local peak latency ( p < 0.01) between stimuli U and R, and no significant differences are observed between the other pairs of illusory stimuli (L&U: p = 0.1032, L&R: p = 0.2137, L&B: p = 0.9897, U&B: p = 0.0522, R&B: p = 0.3535).
To analyze the joint modulation of SSVEP and IVEP stimulation, we compared the EEG responses between SSVEP and hybrid paradigms in the time domain.As depicted in Fig. 4, the amplitude of waveforms are significantly different between hybrid and SSVEP ( p < 0.0001).In hybrid paradigm, the SSVEP signals were superimposed on the IVEP signals.Specifically, for all hybrid stimuli, there were obvious oscillations superimposed on the evoked P2 component, and the N1 component is superimposed only under the R hybrid stimulus condition.In Fig. 5, spatio-temporal features of each hybrid stimulus are visualized through amplitude topographic maps, depicting both negative and positive peaks during the first period of illusory stimulus onset.These maps were created using data from nine electrodes positioned over the parietal and occipital regions.
The aim of the comparison of spectrum amplitude was to assess the influence of the overlaid IVEP against SSVEP.Here, we plot spectrum calculated with fast Fourier transform (FFT) from averaged recorded 5 s data, as shown in Fig. 6.For all stimulus conditions, distinct amplitudes at fundamental  Fig. 6.The spectrum amplitude of average data from all subjects and channels for the SSVEP stimuli and hybrid stimuli flickering at 14 Hz, 13 Hz, 12 Hz, and 11 Hz, respectively.The amplitude of the fundamental frequency and its harmonics for each stimulus frequency are marked with red circles, and the intermodulation frequencies are marked with gray crosses.
frequency and its harmonics of corresponding stimulus frequencies were evoked.The spectrum peaks of hybrid stimuli for 14 Hz and 13 Hz are reduced compared with SSVEP stimuli.However, the amplitude of fundamental frequencies for 12 Hz and 11 Hz are increased with hybrid stimuli.In the comprehensive analysis, no statistically significant differences Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.were observed in the spectral peak values for the fundamental and harmonic frequencies between these two paradigms.To provide a comprehensive assessment, we conducted a comparative analysis of SNR between the hybrid paradigm and the traditional SSVEP paradigm using two distinct SNR estimation methods.Fig. 7(a) illustrates the average SNR values across subjects associated with frequency responses.In the SSVEP paradigm, the SNR of frequency responses is relatively high.The statistical analysis indicates that there are no statistically significant differences in SNR between the two paradigms across four stimulus frequencies: 14 Hz ( p = 0.6331), 13 Hz ( p = 0.2095), 12 Hz ( p = 0.4367), and 11 Hz ( p = 0.6302).Regarding waveform estimation in Fig. 7(b), the hybrid paradigm demonstrates relatively high SNR in terms of mean amplitude within the specified time window.Furthermore, the SNR difference is significant for the L and U stimuli ( p < 0.05 for L stimulus, p < 0.05 for U stimulus).
As the presence of intermodulation components arising from the superposition of illusory stimulus and SSVEP flickering stimulus, here we also evaluated the SNR of two intermodulation frequencies (f±2.5Hz)corresponding to each stimuli, as shown in Fig. 8.

B. Performance Evaluation
During Experiment 2, we compared the conventional SSVEP performance with single flickering pattern stimulus against performance with hybrid stimulus embedded with illusory stimuli for each subject.The classification accuracy was evaluated under the leave-one-out cross-validation procedure.
Fig. 9 shows the classification accuracy of the SSVEP and hybrid paradigms for each subject and the average accuracy over all subjects.It was observed that the accuracy improved gradually with increasing data length for both paradigms.The accuracy results of hybrid paradigm were significantly higher than conventional SSVEP paradigm from 0.2 s to 1.4 s data length ( p < 0.05).Fig. 10 illustrates the ITR (bits/min) with respect to different data lengths for each subject and average ITR across subjects.With statistic analysis, the hybrid paradigm achieved higher average ITR between the 0.2 s to 1.4 s data lengths ( p < 0.05).The maximal average ITR of 44.2 bits/min was obtained at the 0.4 s data length, increasing by 6.9 bits/min compared to the SSVEP paradigm.As for individual results of s4, s6, and s7, there was no improvement in hybrid conditions when compared to SSVEP.Particularly, for subject S6, the SNR values for intermodulation at f −2.5 Hz are 16.18 dB, 15.54 dB, 6.82 dB, and 9.49 dB under four conditions, respectively, and the SNR for f + 2.5 Hz are 13.82 dB, 12.32 dB, 10.29 dB, and 11.72 dB, respectively.According to statistical analysis, for intermodulation frequency, there is no significant difference between SNR of s6 and the mean SNR of the subject group in these four conditions ( f − 2.5 Hz: p = 0.6225; f + 2.5 Hz: p = 0.9559).For the waveform in time course, s6 exhibits low SNR values, measuring at 2.09 dB, 13.90 dB, 3.92 dB, and 4.62 dB, respectively, which were significantly lower than the average subject level ( p = 0.015 < 0.05).
The performance difference between the hybrid and SSVEP paradigms was reduced as the data length increased.To investigate the capacity of the hybrid paradigm to cope with individual differences, we further utilized a one-to-one framework for cross-subject experiment, in which each subject in the source data was utilized for training spatial filters, and the final identification results were generated through a voting strategy [35].Table II presents the cross-subject classification results for each subject in different time windows.
The questionnaire scores for each subject, along with the mean results and standard deviation (SD) are presented in Table III.To visualize the distribution of data, the boxplot

TABLE II CROSS-SUBJECT CLASSIFICATION RESULTS FOR EACH SUBJECT
was provided as depicted in Fig. 11.We conducted Kruskal-Wallis tests to examine the differences in four subjective state evaluations, containing eye fatigue, concentration, physical condition, and mental condition, between the hybrid and SSVEP paradigms.A significant reduction in eye fatigue was observed in hybrid paradigm as indicated by the statistical analysis ( p = 0.0269).In boxplot, the whisker range of hybrid is longer than SSVEP, which represents a higher level of dispersion of scores within subjects in hybrid paradigm.Regarding the mean values of the scores, subjects exhibited higher levels of concentration, and better physical and mental conditions in hybrid paradigm when compared to the SSVEP paradigm.However, none of these three differences were statistically significant (for concentration, p = 0.4196; for physical condition, p = 0.5702; for mental condition, p = 0.0999).

IV. DISCUSSION
Hybrid BCI systems integrate multiple signal modalities, offering enhancing performance compared to single-modality BCIs.Conventional SSVEP-based BCIs have achieved considerable progress and demonstrated high communication rates.However, by combining SSVEP with other EEG signals, the system's performance can be further improved, enabling more efficient and reliable communication for users.In this study, we introduced IVEP to combine with SSVEP, and implemented a hybrid paradigm.For IVEP, the target identification is based on the spatio-temporal characteristics of ERP Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.Fig. 10.The ITR (bits/min) for each subject and average ITR across all subjects of hybrid paradigm and SSVEP paradigm with different data lengths from 0.2 s to 2 s data length with an interval of 0.2 s. components N1 and P2 elicited by different visual illusion stimuli.This study conducted two experiments to evaluate feasibility of the proposed hybrid paradigm.
We first measured the original features of different motion illusion stimuli in a static IVEP paradigm to analyze the distinguishability.As expected, the statistical analysis of overall amplitude waveforms revealed significant differences across different illusory stimuli.For quantification analysis, mean amplitude and local peak latency of N1 and P2 were measured in specified time windows.The local peak latency of N1 component exhibited variations across different illusory conditions, as indicated in Table I.Indeed, it is well-established that the latency and amplitude of ERP components can exhibit independent variations due to various factors [36].This discrepancy may partly be explained by previous research indicating that augmenting stimulus luminance leads to a reduction in peak latency of N1 over occipital region [37].
Note that the mVEP is dominated by the same negative peak between 150 and 200 ms in occipital and occipitotemporal areas, which are commonly designated as N1, N200, or N2 [38], [39].This component is associated with motion processing and may originate in the primary visual cortex (V1) and the middle temporal area (MT) area.A near-infrared spectroscopy (NIRS) study has found the activation of the MT/VT area during motion illusion perception, suggesting that both actual motion and illusory motion are involved in the same brain area [40].Therefore, it is likely that the negative N1 peak observed in our study could possibly be considered as a motion-specific component, given the implication of the relationship between illusory motion perception and real motion perception.
In Experiment 2, we evaluated the performance of the hybrid paradigm and compared it with conventional SSVEP paradigm.From the comparison of grand average waveforms, EEG responses to the hybrid stimuli achieved significantly larger amplitudes than SSVEP stimuli, and IVEP signals can be stably induced after each onset of motion illusion.According to the previous study, it has been demonstrated that the established SSVEP oscillation, which is phase-locked to the onset of flicker, remains stable and is not disrupted  by transient stimulus-induced ERP [41].This suggests that embedded IVEP may not interfere with the oscillatory phase of SSVEP.However, relevant ERP components might be influenced due to the occupied neural oscillation resources by preceding SSVEP [42].For instance, Moratti et al. discovered that N1 latency was delayed and the amplitudes of both N1 and P2 were reduced flickering stimulation [41].In terms of frequency-domain features, the results indicate that the spectral peak for fundamental and harmonic frequency were not influenced significantly by embedded illusion in hybrid stimuli.From comparative analysis of SNR, the SNR of frequency responses do not exhibit significant disparities, indicating comparable SSVEP responses in both paradigms.In addition, when examining the SNR in waveform estimation, we found significant differences for the L and U stimuli.These observations suggest that the hybrid paradigm may exhibit better performance attributed to its additional time-domain characteristics.
In performance evaluation, we validated classification accuracy and ITR, both of which demonstrated hybrid paradigm yielded superior performance.The benefits of performance enhancement diminished as the data length increased.The EEG signals that incorporate multiple features are able to exhibit relatively satisfactory classification performance in shorter stimulation times, which facilitates the reduction of detection time in practical applications.The improvements Fig. 12.The relative deviation of intermodulation frequency SNR and waveform SNR from mean SNR for each subject.For waveform SNR, the deviation is averaged over stimuli, and for intermodulation frequency SNR, the deviation is averaged over stimuli and two intermodulation frequencies.
in performance could be attributed to the detectable and differentiable features provided by IVEP, which added an additional dimension of features in EEG responses.The integration of flickering stimuli with motion illusions effectively enlarged the SNR of EEG responses without significantly compromising frequency features, which facilitated feature extraction through spatial filters.Nevertheless, it is necessary to consider the potential implications of intermodulation frequencies due to their SNR that cannot be disregarded.The intermodulation frequency is the combination of two different stimulus frequencies contained in the stimulus, which reflects the process of integrative perception in the brain [43].From the results of s6, the intermodulation frequency responses is not weak.However, the SNR of waveform related to the IVEP signals is significantly low.It suggests that the performance enhancement in hybrid paradigm may be associated with the IVEP signals.Fig. 12 displays the relative deviation of intermodulation frequency SNR and waveform SNR from the subject mean for each subject.We have found that subjects exhibiting performance improvements tend to demonstrate large positive deviations (s1, s2, s9, s10) or small negative deviations in waveform SNR (s3, s5) relative to the subject average.The waveform SNR appears to correlate positively with performance in the hybrid paradigm.However, intermodulation frequency SNR does not necessarily correlate with performance.Notably, for subjects who exhibit poor performance in SSVEP paradigm (s8, s12), intermodulation Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
frequency SNR seems to be a primary factor for performance improvement in hybrid paradigm.In the classification process, steady-state frequency information and IVEP features are co-extracted as task-relevant components.At present, it is difficult to separate the contribution of IVEP and intermodulation due to the design of periodic superimposition of illusory stimuli, which is a limitation of our experiments.In addition, the capacity of the hybrid paradigm in coping with individual difference was investigated through cross-subject experiments, which have yielded acceptable classification results but not as efficient as subject-specific training results.Hence, considering the individual differences in IVEP, it is essential to explore cross-subject approaches that are optimal for the hybrid paradigm.
The questionnaire results were assessed for four subjective states.Intriguingly, we found a significant difference between the questionnaire scores of eye fatigue, indicating the hybrid paradigm might induce less eye fatigue compared to the SSVEP.According to feedback from part of subjects, this result could be attributed to the non-continuous flickering in hybrid stimuli.However, it is worth noting that the degree of eye fatigue experienced with the hybrid paradigm shows considerable variations between subjects.Furthermore, differences in mean values for concentration, physical condition, and mental condition imply that proposed hybrid paradigm offers a relatively favorable subjective experience.
In hybrid BCI systems, enhanced BCI classification accuracy and reduced brain-command detection time are two of the main objectives [44].Through experimental research, the proposed hybrid BCI paradigm was validated to be more effective and reliable than the single-modality SSVEP paradigm, which is advantageous for communication and control applications.The features of IVEP and SSVEP are co-extracted together by TRCA method, without additional extraction steps therefore does not increase the system cost.It is worth noting that IVEP exhibit distinct characteristics within the narrowband waves, including alpha, beta, theta, and delta waves, as supported by previous studies [12].However, this narrow bandwidth component has not been fully considered.Therefore, the performance could be further improved by exploiting the narrowband frequency information of the IVEP while extracting the harmonics of SSVEP in the hybrid paradigm [45].The lack of our study is that we do not explore the interaction effect of IVEP and SSVEP in experimental study.Since different motion illusions were combined with flickering stimuli of different frequencies, which are considered as variables that need to be controlled.In previous study, Lee et al. found competing effects in the ERP and SSVEP signals while did not directly affect the classification performance [46].
In this study, we utilized four distinct motion illusions in combination with flickering stimuli to establish a BCI system with four available commands.Indeed, this system fulfills fundamental control requirements, such as wheelchair or cursor control.The number of targets plays a crucial role in the practical applications of BCIs due to its close association with communication efficiency and adaptability across various application scenarios.Given that our stimulus interface has not been fully utilized, there is a potential to augment the number of targets while maintaining the same stimulus size configuration.However, in contemplating such an expansion, it is essential to carefully evaluate the distinguishability among illusory targets.In fact, IVEP may be influenced by the visual stimulus properties.Note that visual stimuli of different contrasts can result in different response timing among visual neurons [47].These differences in response timing can contribute to the perception of illusory motion when there are sequential changes in luminance within a visual stimulus.Hence, the motion illusion stimuli need to include pairs of elements with different contrasts with repetitive arrangements to generate illusion.As for the color, it might enhance the illusion perception when compared to gray-scale illusion [48].It is noteworthy that the stimulus size may also influence the elicited IVEP signal, as subjects may not easily perceive motion illusions when the stimulus size is relatively small.To comprehensively investigate the relationship between stimulus size and BCI performance within the hybrid paradigm, it is imperative to conduct experimental studies on this matter.These further studies will not only facilitate the exploration of the upper limit for encoding different illusory targets but also pave the way for expanding the range of potential application scenarios.In addition, performance can be further enhanced by modulating the onset time for each motion illusion in hybrid paradigm in order to expand the differences of temporal features between targets.

V. CONCLUSION
In this study, we presented a novel hybrid paradigm that combined IVEP and SSVEP to improve performance in four-command task.The motion illusions were embedded in steady-state flickers so that the IVEP and SSVEP can be evoked simultaneously.The results from Experiment 1 validated the difference among motion illusions in time domain.In comparison to the conventional SSVEP in Experiment 2, the proposed hybrid paradigm outperformed in classification accuracy and ITR in short time window.The introduction of illusion significantly enhanced the SNR of EEG responses, thus facilitating spatial filter in feature extraction phase.Through improving classification accuracy and reducing detection time, the proposed paradigm could alleviate the users' burden of usage.Meanwhile, the subjective questionnaire results indicate that hybrid BCI system with non-continuous flickering stimulation with motion illusion might be able to reduce eye fatigue to some degree.Further research in IVEP-SSVEP hybrid BCI is a worthy endeavor, and has the potential to further improve the performance of BCI systems for practical applications.

Fig. 1 .
Fig. 1.(a) Stimulation design for IVEP paradigm, containing four illusory stimuli to induce the perception of illusory motion.(b) Stimulation design for SSVEP paradigm.Four frequencies are 14 Hz (left), 13 Hz (up), 12 Hz (right), and 11 Hz (bottom) with a 0.5 π phase interval.(c) Stimulation design of each hybrid stimulus in each frame for one second with 60 Hz refresh rate.Cosine modulation of luminance from 0 to 1 represents dark to bright.The gray shaded area indicates the time window for the onset of the illusory stimuli.(d) The display protocol of hybrid paradigm in one trial.The illusory stimuli were periodically presented after SSVEP stimuli onset for 200 ms.The SOA was 400 ms where illusory stimuli are presented for 50 ms.

Fig. 4 .
Fig. 4. The grand average waveforms of hybrid and SSVEP stimuli in 2 s time window after stimulation onset.The vertical solid lines denote the motion illusions onset, and the gray rectangular filled areas are its presentation period (50 ms).The filled fields along waveforms indicate the SEM.

Fig. 5 .
Fig. 5.The amplitude topographic maps of negative and positive peaks during the first period of illusory stimuli onset in the hybrid paradigm.Nine electrodes overlaying parietal and occipital areas are involved.

Fig. 7 .
Fig. 7. (a) Average SNR of frequency response for each stimulus across subjects.(b)Average SNR of waveform for each stimulus across subjects.Error bars = ± SEM.

Fig. 8 .
Fig. 8.The average SNR of intermodulation frequencies for four stimuli in hybrid paradigm.Error bars = ± SEM.

Fig. 9 .
Fig. 9.The classification accuracy for each subject, and average accuracy across all subjects of hybrid paradigm and SSVEP paradigm with different data lengths from 0.2 s to 2 s data length with an interval of 0.2 s.

Fig. 11 .
Fig. 11.The boxplot of subjective evaluation of SSVEP and hybrid paradigms concerning the eye fatigue, concentration, physical and mental conditions for participants.

TABLE III QUESTIONNAIRE
RESULTS FOR SSVEP AND HYBRID PARADIGMS