The Effects of Bilateral Phase-Dependent Closed-Loop Vibration Stimulation With Motor Imagery Paradigm

Vibration stimulation has been shown to have the potential to improve the activation pattern of unilateral motor imagery (MI) and to promote motor recovery. However, in the widely used left and right hand MI brain-computer interface (BCI) paradigm, the vibration stimuli cannot be directly applied to the imaginary side due to the spontaneity of imagery. In this study, we proposed a method of phase-dependent closed-loop vibration stimulation to be applied on both hands, and explored the effects of different vibration stimuli on the left and right hand MI-BCI. Eighteen healthy subjects were recruited and asked to perform, in sequence, MI tasks under three different conditions of vibratory feedback, which were no vibration stimulus (MI), phase-dependent closed-loop vibration stimulus (PDS), and continuous vibration stimulus (CS). Then the performance of the left and right hand MI-BCI and the patterns of brain oscillation were compared and analyzed under these different stimulation conditions. The results showed that vibration stimulation effectively boosted the activation of the sensorimotor cortex and enhanced the functional connectivity among sensorimotor-related brain regions during MI. The closed-loop stimulation evoked stronger event-related desynchronization patterns on the contralateral side of the imagined hand compared to continuous stimulation. There was a more obvious distinction between left hand task and right hand task. In addition, phase-dependent closed-loop vibration stimulation increased classification accuracy by approximately 7% (paired t-test, p=0.004, n=18) compared to MI alone, while continuous vibration stimulation only increased it by 4% (paired t-test, p=0.067, n=18). This result further demonstrated the effectiveness of the phase-dependent closed-loop vibration stimulation method in improving the overall performance of the MI paradigm and is expected to be further applied in areas such as stroke rehabilitation in the future.


E LECTROENCEPHALOGRAM (EEG) is a signal com-40
monly used in non-invasive brain-computer interface 41 (BCI) systems, which can help people use brain activity to 42 communicate and interact with external devices [1]. As an 43 active BCI, the motor imagery (MI) paradigm is widely used 44 in robotic control, neurorehabilitation training, and other fields. 45 It allows users to modulate the alpha/beta rhythm of the 46 sensorimotor cortex by imagining the movements of limbs 47 such as hands or feet. Therefore generating patterns similar 48 to the ones induced by active movements in the electromag- 49 netic field. Such cortical activities are termed event-related 50 (de)synchronizations (ERD/ERS) [2], [3], [4], [5]. In recent 51 years, MI has also often been used to enhance motor learning 52 and restore motor functions. 53 However, the factors such as individual differences, training 54 time, and etc. strongly influence the decoding accuracy of 55 MI-BCI [6] with about 15-30% of users unable to use MI-56 BCI (BCI-illiteracy) [7], [8]. These problems often affect the 57 overall performance of motor imagery and significantly limit 58 the application of the MI paradigm. Therefore, some studies 59 have been conducted to improve the overall performance of 60 the motor-imagery paradigm. In addition to the approaches 61 of the signal processing algorithm improvements, the feature 62 and electrode channel selection, and the multi-signal fusion 63 methods (fNIRS, EMG, etc.) [9], [10], [11], [12], [13], [14], 64 [15], [16], [17], [18], the direction of combining multiple sen-65 sory stimuli with the MI paradigm to improve the overall BCI 66 performance and creating a new fused BCI is also working on 67 the right track [19]. This direction focuses on a hybrid BCI 68 that combines MI paradigms with visual and haptic sensory 69 stimuli or simultaneously induced multiple brain modalities 70 (e.g., SSSEP, SSVEP) [20], [21], [22], [23], [24]. Among 71 them, the haptic channel does not occupy the visual channel 72 and thus retains the advantage of MI's spontaneity. It allows 73 the subject to modulate brain activities autonomously so it is 74 more practical, especially for users with vision deficits or in 75 visually occupied scenarios. effect of MI when simultaneously applying tactile stimuli on 135 both sides, and existing studies have also yielded mixed results 136 [23], [25], [27], [42]. Our previous study found that when 137 applying continuous vibration stimulation to either imagery or 138 resting task of a non-dominant hand, it produced significant 139 activation in contralateral sensorimotor areas. This has atten-140 uating the difference in ERD patterns between the imagery 141 task and the resting task, and affecting classification accuracy. 142 In contrast, closed-loop vibration stimulation produced fewer 143 adverse effects and enhanced overall MI performance more 144 efficiently. This may also be valid for the left-and right-handed 145 MI paradigm.

146
The main contributions of this paper include: in order 147 to further investigate the effect of phase-dependent closed-148 loop vibration stimulation on the overall performance of the 149 motor imagery paradigm, we applied vibration stimulation to 150     This study adopted two modes of vibration stimulation: con-221 tinuous vibration stimulation and phase-dependent vibration 222 stimulation. In order to stimulate the Pacinian and Meissner 223 corpuscles simultaneously, and these two mechanoreceptors 224 are sensitive to frequencies above 100 Hz and 20-50 Hz [46], 225 we applied the continuous vibration stimuli constantly during 226 the imagery period (5th-11th seconds of each trial). An elec-227 trical signal of 200 Hz sinusoidal carrier frequency modulated 228 by 23 Hz sinusoidal frequency was generated using a computer 229 soundcard and amplified using an audio amplifier in order to 230 drive the actuators. The phase-dependent vibration stimulation 231 determines when to trigger the vibration stimulus depending 232 on the real-time phase predicted by the instantaneous phase 233 prediction algorithm. Each triggered vibration at a frequency 234 of 200Hz for 20ms. The stimulus was applied to the falling 235 ([5 × π/6, 4 × π/3]) phase of the alpha oscillations in the C4 236 channel. To avoid repeated triggering or estimation errors, the 237 interval between each stimulation must be greater than 80ms. 238 In order for the subjects to feel the rhythm better, the amplitude 239 of every third vibration stimulus is 50% higher than the first 240 two ("tic-tic-toc" pattern [47]).

241
Before starting the experiment, each subject was asked to 242 feel the intensity of the vibration at different amplitudes and 243 to choose the intensity that they could clearly feel without 244 affecting their imagination as the parameter for the experiment. 245 Vibration amplitudes were controlled by the audio amplifier. 246

D. Experimental Procedure 247
The experimental scene is shown in Fig. 1. The subjects sat 248 in a comfortable chair throughout the whole process. Their 249 eyes were about one meter away from the monitor, and their 250 hands were relaxed on the armrests of the chair. To avoid 251 placebo effects, the piezo actuators were attached to the sub-252 jects' wrists from the beginning to the end of the experiment. 253 The experiment was controlled by Psychtoolbox [48].

254
The experiment consisted of three sessions: motor imagery 255 without stimulus (MI), motor imagery with phase-dependent 256 vibration stimulus (PDS), and motor imagery with continuous 257 vibration stimulus (CS). The subjects executed the three ses-258 sions in sequence. Fig. 2 illustrates the paradigm of a single 259 trial in every session. The time structure of all sessions was 260 the same as the imaginary task performed by the subjects, but 261 the vibration applied were different.

262
At the beginning of each trial, a white cross was displayed 263 on the screen, and the subjects could relax and rest for 264 4 seconds. At the fourth second, a white circle appeared in the 265 middle of the cross and lasted 1s, reminding the subjects to 266 pay attention as the imagining task was about to start. Within 267 5-11 seconds, left or right arrows randomly appeared on the 268 cross, and the subjects performed the corresponding left-hand 269 or right-hand MI tasks according to the direction of the arrow. 270 In the end, the screen displayed the white cross again, and the 271 subjects relax and rest before entering the subsequent trial. 272 To minimize artifacts such as electrooculography, subjects 273 were asked to avoid extra body movements such as blinking 274 during the motor imagery tasks. In this study, we used custom-built MATLAB scripts and 287 EEGLAB toolbox algorithms to analyze EEG data offline.  The phase-locking value was used for measuring the phase 308 synchronization information between pairs of signals, which 309 can quantify the functional connectivity between different 310 brain sites. It is calculated over the N-sample window as 311 follows: where N represents the sample amount of each signal. ϕ x (t)− 314 ϕ y (t) stands for the instantaneous phase difference between 315 each pair of EEG channels (x, y) on time window t. The 316 instantaneous phase is calculated by the Hilbert transform. 317 The PLV ranges from 0-1, with 1 denoting complete phase 318 synchronization and 0 denoting that the signals are com-319 pletely desynchronized. We used a sliding window of 1 s to 320 average the data during the imagery task to obtain a 63 × 321 63 PLV matrix. Furthermore, we investigated the local-scale 322 synchronies of left and right M1 areas as these areas are 323 considered as the primary cortical areas involved in the hand 324 MI tasks. For the measurement of local-scale synchrony, four 325 neighboring electrodes of C3 and C4 were combined to form a 326 five-electrode group. Averaging four combinations of electrode 327 pairs from five electrodes yielded the phase-locking value.

328
In addition to the time-frequency feature, classification 329 accuracy is also a crucial indicator for the overall performance 330 of MI-BCI. We used the bandpass filter to filter the raw data of 331 all 63 channels to 8-30 Hz and extracted the 5th-10th second 332 (5 s after the beginning of imagination) data of each trial for 333 feature extraction and classification. To investigate the impact 334 of vibration stimulation on the classification accuracy of the 335 MI paradigm, we use the traditional common spatial pattern

392
In the spatial domain, the enhancement of the ERD pat-393 tern by vibration stimulation can be clearly observed. While 394 comparing to the CS task, the activation area of the PDS 395 task is more concentrated, and the activation degree is the 396 deepest. The MI and CS tasks produced more significant 397 contralateral activation in the right-handed imagery task than 398 the left-handed task. However, the PDS task was the opposite, 399 phase-dependent vibration stimulation significantly enhanced 400 the activation of the cortex near C4. No concentrated ERS can 401 be seen from the topographic map, but the energy enhancement 402 on the ipsilateral side can still be clearly observed. It can be 403 seen that bilateral tactile stimulation can boost the activation 404 of the motor-sensory cortex induced by motor imagery. alpha and beta bands, with the CS being mostly higher than 412 the PDS in terms of mean values. There were significant 413 differences between C3 local and C4 local within all the same 414 frequency bands and tasks (paired t-test, p<0.01). In contrast, 415 the differences between the left and right hand tasks were not 416 significant under the same conditions and regions. Regarding 417 mean values, there was almost no difference between the left 418 and right hand tasks for CS, except for the C4 local in the beta 419 band. In contrast, there were significant differences between 420 the MI and PDS tasks, especially in the C4 local. That may 421 be because bilaterally applied vibration stimuli significantly 422 induced cortical activation on both sides. Vibratory stimuli 423 elicited more pronounced changes in PLV values in the beta 424 band compared to the alpha band because beta oscillations are 425 particularly sensitive to somatosensory stimuli. 426 From the PLV matrix in Fig. 6, CS and PDS have more 427 electrode pairs to obtain larger PLV values than MI. Vibration 428 stimulation activated more brain regions in the alpha band. 429 It can be seen that PDS has more synchronized nodes and 430 the highest level of brain activation, with greater overall 431 synchronization than MI and CS. In addition, the right-handed 432 task produced slightly higher levels of synchrony than the 433 left-handed task in the PDS, but the difference between the 434 left-and right-handed tasks was not significant in the MI 435 and CS.  Fig. 7 illustrates the offline classification accuracy of all 438 subjects in different conditions. The average classification 439 accuracy of the three tasks was 63.6%, 67.4%, and 70.2%, 440 respectively. Except for subjects S15 and S18, the addition 441 of vibration stimulation effectively improved the classification 442 accuracy of MI. Compared with MI, CS and PDS increased 443 by approximately 4% (paired t-test, p=0.067) and 7% (paired 444 t-test, p=0.004), respectively. This result demonstrated that 445 Fig. 3. Grand-averaged spatial distributions of ERSP patterns of all subjects for each class and frequency band. The upper and lower rows correspond to left-hand and right-hand MI tasks, respectively, and each column corresponds to three typical frequency ranges in different sessions.    sensory-motor areas of the brain. Chatterjee et al. [19] found 467 that when vibration stimulation applied on the ipsilateral side 468 of the imagined hand, BCI accuracy could be significantly 469 improved, with the left side showing significantly higher 470 improvement than the right side. However, because MI is a 471 spontaneous BCI, it is not possible to apply the vibration 472 stimuli only on the imagined side, so the stimuli are often 473 applied to both sides simultaneously.

474
From the available findings, it appears that directly applying 475 of the vibratory stimulus to both sides does not significantly 476 improve the overall performance on the left and right hand 477 MI tasks. Because MI is a complex mental task, cortical 478 activation due to the perception of tactile stimuli alone does 479 not necessarily result in an immediate increase in classifica-480 tion accuracy [49]. In this study, the CS task significantly 481 enhanced both bilateral motor-sensory cortical activation and 482

517
It has been shown that the tactile afferent input provided by 518 vibration stimulation can increase the motor-related cortical 519 excitability of subjects [51]. A similar phenomenon can be 520 observed from Fig. 3 and Fig. 4. Combining the average ERD 521 fluctuation from Fig. 8 and the peak ERD from Fig. 9, the 522 left-hand MI task benefits more from the somatosensory stim-523 uli. Regardless of whether the left or right hand was imagined, 524 the vibratory stimuli generally enhanced the desynchronization 525 of the C4 channel, and the ERD amplitude of the contralateral 526 channel was significantly higher when the left hand was imag-527 ined than the right hand. Notably, the addition of the vibration 528 stimulation resulted in enhancing ERD in the motor-sensory 529 area contralateral to the imagined hand while also producing 530 energy suppression on the ipsilateral side. That is a similar 531 result observed in many studies that have applied tactile stimuli 532 to both hands [23], [25], [27]. Although some studies have 533 shown that ERD are often produced on the ipsilateral side 534 with left hand MI [26], this desynchronization observed from 535 the ipsilateral side of both tasks were certainly deriving from 536 the activation of vibration stimuli. This phenomenon causes a 537 reduction in the specificity of the ERD pattern in the left and 538 right hand MI paradigm. At the same time, this phenomenon 539 is also reflected in synchronization, as the p-values of the PLV 540 in Table. I reflect that the CS produces almost no difference 541 in PLV between the left and right hand tasks in the same 542 frequency band and brain region. That is one of the reasons 543 that in many studies the simultaneous application of vibration 544   Fig. 8, where the PDS task produced persistent deepest ERDs 562 in the sensorimotor area contralateral to the imagined hand, 563 these ERDs is significantly stronger than the ones produced 564 by CS task. In contrast, the ERD waveforms on the ipsilateral 565 side of the imagined hand were not significantly different from 566 the CS task.

567
Because MI shares some resources with the neural response 568 network of tactile perception, sustained vibratory stimulation 569 generates competition for resources. In turn, MI affects the 570 perception of vibration stimuli, and it has been shown that 571 MI significantly inhibits the phase synchronization of the 572 ipsilateral SSSEP. It is found in Fig. 5 that, although the CS 573 obtained the highest PLV means in the beta band of C3 local, 574 it did not differ significantly from the MI due to the large 575 disparity in response to beta frequency stimuli across subjects. 576 Compared to both the MI and CS tasks, PDS task significantly 577 improved the general enhancement of MI. Therefore, the MI 578 with the continuous vibration stimulation, the closed-loop 634 vibration stimulation reduced interference with imagery tasks 635 while promoting deeper and more sustained activation in the 636 bilateral sensorimotor cortex. It can more efficiently combine 637 sensory input with motor imagery and enable closed-loop 638 vibration stimulation to significantly improve the classification 639 accuracy of MI-BCI. With simple equipment, less preparatory 640 work, and high user acceptance, the phase-dependent closed-641 loop vibration system can be applied to assist stroke rehabili-642 tation training or benefit people with complete somatosensory 643 systems but impaired motor functions.