By Topic

IEEE Quick Preview
  • Abstract

SECTION I

BACKGROUND

RECENTLY, several studies have investigated the effect of ongoing oscillatory activity preceding an event on subsequent processing and task outcome performance. The reported effects vary extensively according to the investigated task. Numerous visual perception electroencephalography (EEG) studies link improved performance in stimulus detection to decreased amplitude in the alpha frequency band (around 7–13 Hz) [1], [2], [3], [4], [5], [6] over parieto-occipital areas. Additionally, [3] showed that trials with perceived stimuli are characterized by a lower phase coupling in the alpha band and a higher phase coupling in the beta (12–20 Hz) and gamma (20–45 Hz) frequency bands. Similarly, lower pre-stimulus synchronicity as well as lower pre-stimulus amplitude of the parieto-occipital alpha rhythm were found to be associated with faster reaction times in an auditory oddball task [7] and an intramodal (visual–auditory) task [8], respectively. On the other hand, better cognitive performance was obtained after artificial enhancement of the baseline alpha activity (around 10 Hz) by transcranial magnetic stimulation (TMS) on parietal and frontal areas in a mental rotation task [9], [10], and by audio-visual flickering in a visuo-spatial motor task [11]. Also, large alpha power was associated with higher performance and stronger event-related desynchronization (ERD) in memory [12], [13] and visual processing [14], [15] tasks.

Ongoing oscillations in the somatosensory cortex and their influence on performance have also been analyzed in perception studies. In those studies, performance is defined as the ability to consciously detect weak somatosensory stimuli and measured in terms of reactions times. However, the results linking performance and ongoing oscillations are not always congruent in the literature. In [16], better performance was associated with intermediate pre-stimulus magnetoencephalogram (MEG) amplitudes over the sensorimotor cortex at around 10, 20, and 40 Hz, and with larger amplitudes over the parietal region at 10 and 20 Hz. A second study on the same data [17] reported improved stimulus detection in trials with higher power in the alpha (7–14 Hz) and beta (14–29 Hz) frequency bands over the somatosensory cortex. Lange et al. [18] reported better performance again for intermediate alpha (congruent with [19]) but also for low beta amplitudes. Additionally, later somatosensory evoked potentials (SEP), known to be linked to a higher degree of cognitive processing [20], [21], were found to be positively correlated with pre-stimulus alpha amplitude in an online EEG study [22].

Finally, in visuomotor pattern discrimination tasks, high pre-stimulus EEG beta (14–30 Hz) power in the prefrontal cortex, low pre-stimulus EEG beta power in the sensorimotor cortex, and alpha and beta power in the occipital and temporal areas were associated with faster reaction times [23], while high alpha amplitudes in parietal regions were hypothesized to reflect top-down inhibitory suppressing of distracting information [24].

Altogether, pre-stimulus alpha activity in parieto-occipital areas is usually proposed as a modulator of attentional states; either by enhancing visual attention by lower amplitudes [1], [3], by putting functional networks in an idling or resonance state [10], [16], or even by inhibiting task-irrelevant brain areas [24]. Similarly to the occipital visual alpha, which is suppressed by visual processing [25], the sensorimotor rhythm (SMR), which can involve only alpha, only beta, or both, is suppressed during motor preparation and execution [26], [27], [28], motor imagination [29], [30], motor observation [31], [32], and somatosensory stimulation [33]. Thus, a functional role similar to the parieto-occipital alpha can also be hypothesized for the pre-stimulus SMR. Still, the role of pre-stimulus sensorimotor oscillations is much more difficult to assess, first because of contradictory results in the existing literature [16], [17] and second because the influence of pre-stimulus beta SMR, which can be stronger than the alpha influence in some cases [18] (and rarely also of pre-stimulus gamma), suggests a more complicated network functionality than it is hypothesized for the parieto-occipital areas [16].

In this study, we investigate the role of the pre-stimulus EEG SMR in a classical SMR-based brain–computer interface (BCI) experiment. SMR-based BCIs make use of voluntary SMR modulations usually measured by EEG signals [34], [35], [36]. These can be induced in a well-defined way by left and right hand motor imagery [37]. The corresponding decrease/increase in the SMR amplitude is called event-related desynchronization/synchronization (ERD/ERS). Typically, during the imagination of a hand movement, an ERD occurs over the cortical area in the contralateral hemisphere, while an ERS is observed over the cortical area in the ipsilateral hemisphere as well as the noncorresponding areas in both hemispheres [38], [39]. This was described as the focal ERD/surround ERS phenomenon [38], [40].

The power spectra measured by EEG exhibits a characteristic Formula$1/f$ shape (where Formula$f$ is the frequency), and typically one or two peaks around 10 and 20 Hz. The distance of these peaks from the Formula$1/f$ curve corresponds to the degree of synchronization of the cortical networks at rest, matching the possibility of modulating the rhythmic activity by motor imagery, i.e., to desynchronize the underlying networks and suppress the peaks. With suitable signal processing and machine learning techniques [41], [42], [43], this phenomenon (depicted in Fig. 1) allows a BCI system to distinguish the two imagined hand movements in each trial, and thereby to provide a control signal for an electronic device.

Figure 1
Fig. 1. SMR amplitude modulations during motor imagery. ERD during motor imagery of the left (purple) and the right (green) hand. The left and right panels show the attenuation of the spectral power in the SMR band(s) (8–14 and 13–28 Hz) following motor imagery in two Laplacian filtered channels over the left (C4 lap) and the right (C3 lap) hemisphere. The central panel shows scalp topographies of these spectral attenuations. After imagery of the left hand, an ERD is observed over the right hemisphere and the opposite is observed for the right hand. Figure taken from [44]—reprint permission to be obtained.

This approach has been demonstrated to be effective for applications like communication [45], [46], navigation systems [47], [48], and neuroprosthetics [49], [50], [51]. The viability of SMR-based BCIs was verified for patients diagnosed with spinal cord injury [50], [52], tetraplegia [49], and amyotrophic lateral sclerosis [53].

The classical SMR-based BCI experiment employs a visuo-motor paradigm, where the users imagine to move their left or right hand, depending on the direction indicated by a visual cue. The BCI performance is measured in terms of the accuracy of a classifier trained to distiguish left and right hand trials based on post-stimulus activity. The translation of BCI technology to everyday life is challenged by a strong variability in BCI classification performance between the users, but also between sessions and even trials for the same user [54]. In fact, the motor imagery task requires a high level of concentration, relaxation (while staying awake), and patience, and can be very challenging. For this reason, good performance relies on multiple factors including concentration and motor skills [55]. Still, following [44], it can be assumed that users with high SMR amplitude at rest, e.g., measured before the experiment starts, are more likely to perform well. Therefore, we hypothesize that part of the inter-trial performance variability can be explained by the strength of the pre-stimulus SMR, and that trials with higher pre-stimulus SMR amplitude can be better classified than trials with low SMR amplitudes. Under this assumption, a high SMR amplitude over both sensorimotor areas preceding motor imagery would indicate an idle or even inhibitory state of the sensorimotor system in the same way high parieto-occipital alpha signifies relaxation of the visual system. This should allow better stimulus processing and thus a stronger ERD over the task-related hemisphere, inducing a more clear focal ERD/surround ERS pattern [38], [39], [40], which is easier to classify.

To investigate this hypothesis, we explore the influence of the SMR amplitude in the pre-stimulus interval on the quality and dynamics of the ERD/ERS patterns, as well as the induced changes in classification performance. Additionally, in order to ensure that the variation in discrimination ability is an effect of the SMR amplitude in the pre-stimulus interval, and not a consequence of the specific task sequence, we separately analyze the trials preceded by a trial associated with the same task (right–right or left–left) and the trials preceded by a trial associated with the other task (left–right or right–left).

SECTION II

MATERIAL AND METHODS

A. Experimental Setup

We consider data formerly presented in a large-scale BCI study [44], from which we select all those 23 naive participants who reached at least 70% accuracy in left versus right hand BCI feedback (15 females; mean age Formula$26.7\pm 12.2$ years). The participants sat in a comfortable chair with arms lying relaxed on armrests. Each dataset was acquired during a single BCI session consisting of a calibration and a feedback phase. During the calibration, a visual stimulus in form of an arrow indicated which type of imagery was to be performed for the coming 4 s [see Fig. 2(a)]. Each trial lasted for 8 s. Three runs of 25 trials (75 trials) were recorded. Here, no feedback was provided to the participant. An offline training of the feature extraction/classification algorithms followed. The trained system was then used to calculate the online feedback during the succeeding feedback session. Feedback was provided to the participant after 1 s of stimulus presentation in form of a purple cross moving according to the realtime classifier output. The stimulus was shown for a total of 5 s and online feedback was provided during the 4 last s. After that, the cross froze in the end position showing the final result for 2 s. A whole trial lasted 9 s [see Fig. 2(b)]. Three runs of 50 trials per class (300 trials) were recorded.

Figure 2
Fig. 2. Trial design: (a) a calibration and (b) a feedback trial. Figure taken from [44]-reprint permission to be obtained.

Brain activity was recorded from the scalp with multichannel EEG amplifiers (BrainAmp DC by Brain Products, Munich, Germany) using 119 Ag/AgCl electrodes (reference at nasion; manufacturer EasyCap, Munich, Germany) in an extended 10–20 system. EEG data were sampled at 1000 Hz and band-pass filtered between 0.05 and 200 Hz.

B. Feature Selection and Classifier Training on Calibration Data

Data from the calibration session were low-pass filtered at 45Hz and subsampled at 100 Hz. A variance-based automatic artifact rejection was applied to reject trials and channels with evident post-stimulus amplitude abnormalities. This method successively rejected each trial with a standard deviation higher than twice the mean overall standard deviation (with rescanning until no more trials were rejected). Continuous data were then band-pass filtered and epoched using the subject-specific frequency band and time interval chosen during the experiment, and used for the online feedback. The selection of these parameters was done during the experiment with a semi-automatic procedure using heuristics to maximize the discriminability of the two classes during the motor imagery task, i.e., in the post-stimulus interval. Details about this procedure are described in [56]. The distribution of the selected frequency bins in the range in which the heuristic was applied (1–34Hz) is depicted in Fig. 3(a). Common spatial patterns (CSPs) analysis was conducted, resulting in up to six subject-specific spatial filters. CSP filters maximize the difference between the variance of the CSP filtered data for the two classes. Usage of CSP allows one to rank filters by their amount of class related information and thereby perform filter selection. Here, this is done based on heuristics described in [56]. Up to three filters per class were selected. Features for classification were obtained by calculating the log-variance of the CSP filtered data and used to train a linear discriminant analysis (LDA) classifier.

Figure 3
Fig. 3. Subject specific parameters. (a) Distribution of the subject-specific frequency bands. For each frequency bin, the histogram indicates the number of subjects for whom the corresponding frequency was included in the selected frequency band. (b) Selection abundance of each channel as best SMR channel. For each channel, the number of subjects for whom that channel was selected is color coded.

C. Grouping of Feedback Session Trials

Data from the feedback session were low-pass filtered at 45 Hz, subsampled at 100 Hz and band-pass filtered as well as epoched using the subject-specific frequency band and time intervals. Next, local average reference (LAR) spatial filtering was applied in order to enhance local signal-to-noise ratios. That is, the activity in each channel was reduced by the average activity in the eight closest neighboring channels (i.e., for the derivation of C3, the channels FC3, CFC5, CFC3, C5, C3, C1, CCP5, CCP3, and CP3 were involved). For each hemisphere, the LAR filtered channel with the highest Formula$r^2$ score among those belonging to the motor areas was selected, which resulted in two subject-specific best SMR electrodes. The chosen SMR channels are depicted in Fig. 3(b), where the selection abundance (number of subjects) for each channel is color coded.

The logarithm of the band power during the 1000 ms pre-stimulus time interval (Formula${-}1000$ to 0 ms) averaged over these two channels, indicated in the following as SMR Pow, was calculated as a benchmark quantity. Trials were divided into two groups, depending on their SMR Pow score. The low-group contains the trials in which the SMR Pow is lower than the 40th percentile, while the high-group contains the trials in which the SMR Pow is higher than the 60th percentile.

In order to ensure that differences in the two groups are not due to the type of motor imagery performed in the previous trial instead of the pre-stimulus SMR Pow score itself, all trials from high and low groups were merged again and re-separated according to the previous trial type: trials were put into the same-group, if the previous motor imagery type was the same, and into the diff-group, if the previous motor imagery type was different. This yielded well balanced groups: on average, the same subgroups (samelow and samehigh) each consisted of 56 ± 10 trials, and the diff subgroups (difflow and diffhigh) each consisted of 58 ± 11 trials. Both the high group and the low-group consisted of 114 ± 19 trials. Furthermore, the high group consisted of 50.1 ± 4.3%same trials and 49.9 ± 4.3 % diff trials, and the low group comported 50.2 ± 3.4% same trials and 49.8 ± 3.4% diff trials.

D. Classification of Feedback Data

For each group, the band-pass filtered data were spatially filtered using the CSP filters calculated on the calibration data. Continuous sequences of CSP features were obtained by extracting segments of 1000 ms duration in a sliding window with a step size of 50 ms, and by calculating the log-variance of the CSP filtered data within these segments. This was done to analyze the time course of the influence of pre-stimulus SMR on the post-stimulus classification. This procedure differs essentially from the online classification, where the classifier is evaluated every 40 ms, and classifier outputs are integrated over time such that the final classification is a result of the overall classification in the post-stimulus time interval. In contrast to that, no information about classifier outputs in preceding time windows was used here.

All features were classified by the previously trained LDA, resulting in a classification accuracy for each time window. Based on the classification scores, trials were separated into trial subgroups for visualization.

E. Grand Average ERD

To calculate the time-varying SMR band amplitude, the envelope of the band-pass filtered LAR derivations of all channels was calculated using a Hilbert transform and smoothened with a moving average filter (200 ms window). Epochs from 2000 before to 7000 ms after the cue presentation were extracted. Baseline correction was applied by subtracting the average activity in the entire time segment separately for each trial and channel. This was preferred to subtracting activity in a pre-stimulus interval, since we are here exactly interested in differences in activity in the pre-stimulus interval. No spatial filtering was applied in order to facilitate neurophysiological interpretation of the resulting scalp topography plots.

F. Significance Test

The differences between conditions was calculated by means of the point biserial correlation coefficient, which quantifies the dependence of the SMR band amplitude on trial group membership. A correlation score was computed for each participant. Whether that score was significantly different from zero was assessed by means of Fishers Formula$z$ transformation. Grand-average Formula$z$-scores were calculated by aggregating the participant-wise Formula$z$-scores, and Formula$p$-values for the null hypothesis of zero correlation in the grand average were computed by means of two-sided Formula$z$-tests.

To dissociate the effect of pre-stimulus SMR power from the effect of the preceding trial, we furthermore performed an analysis of variance (ANOVA) using a 2×2 factorial design with SMR Pow (low, high) and the previous trial type (same, diff) as factors, and BCI classification accuracy as the dependent variable.

SECTION III

RESULTS

A. Classification Error

Fig. 4(a) shows the grand average classification error for all 23 subjects for the two groups of trials high pre-stimulus SMR (green) and low pre-stimulus SMR (purple) built depending on the pre-stimulus SMR band power (SMR Pow). The time axis in Fig. 4 represents the timing of one trial and the end of the 1000 ms classification window. Thus, at Formula${t}=0$, i.e., when the cue is presented, the classification error is calculated on the 1000 ms pre-stimulus time window while at Formula${t}=1000$, i.e., when the cross starts to move and feedback is given, the classification error is calculated in the 1000 ms window after cue presentation, but still without feedback. The color coded Formula$p$-values resulting from the ANOVA for each time window are represented in the pink colorbar on the bottom of each figure. Similarly, Fig. 4(b) shows the grand average classification error for all 23 subjects for the two groups of trials different previous trial (green) and same previous trial (purple) built depending on the previous trial type. Significant differences (i.e., when Formula$p < 0.05$ in the bottom bar) between trial groups can be seen in both settings. However the effect varies in size and timing. The classification error was lower for the high compared to the low group over the whole trial length. This decrease is significant in an interval ranging from approximately 500–3000 ms after the appearance of the cue, i.e., during the main ERD time window and more importantly during the online feedback. On the contrary, even if much more significant, the difference between the classification error for the diff and the same group is located earlier in the trial, already much before the cue onset and until approximately 1500 ms, i.e., outside the online feedback. In fact, in that interval the classification error is still high and it is obviously an artifact of the type of division, which includes the class information.

Figure 4
Fig. 4. Grand average classification error in 1000 ms windows (50 ms overlap) for trial groups. The Formula$x$-axis represents the timing of one trial and the end of the classification window (Formula$t=0$ is the cue onset and the corresponding error is calculated on the pre-stimulus 1000 ms; at Formula$t=1000$ ms the online feedback starts). Formula$p$-values are color coded and shown in the pink bar on the bottom. (a) High (green) versus low (purple) pre-stimulus SMR. (b) Different (green) versus same (purple) previous trial.

Fig. 5 shows the Formula$p$-values obtained for each classification window and for both comparisons (blue: high versus low and orange: diff versus same) and their interaction (gray). It can be seen that a significant interaction between both separation factors is only present in an interval ranging approximately from 250 ms before the cue onset to 1750 ms, which corresponds to the early phase of the trial.

Figure 5
Fig. 5. Evolution of ANOVA Formula$p$-values for the effects of pre-stimulus SMR (high versus low in blue) and previous trial (diff versus same in orange), as well as the interaction (gray) for each classification window across the whole trial. The x-axis represents the end of the classification window as in Fig. 4. The two dotted lines represent the significance levels of Formula$p = 0.01$ and Formula$p = 0.05$.

B. Analysis of Group Effect on SMR

Fig. 6 is a histogram plot displaying the number of consecutive trials belonging to the same group (low-group in green, red-group in red, middle group in black). It can be seen that, for 123 out of 228 trials (more than 50%), the SMR state (high or low) changes after one trial, for 20% of the trials, the SMR state changes every two trials and for only 10% of the trials the SMR state stays the same for three consecutive trials. Moreover, the number of consecutive trials is the same for the two groups. Thus, we can assume that the SMR amplitude fluctuates fast, and that neither the high nor the low SMR state prevails for a longer time.

Figure 6
Fig. 6. Number of consecutive trials belonging to the same group. Green: high-group, red: low-group, and black: unused trials.

SMR band power mean and standard deviation for low (red) and high (green) group for each participant are depicted in Fig. 7. High variability among subjects and also between the groups can be observed. In general, the high-group tends to have a higher variability than the low-group. Very well performing subjects exhibit a higher difference between high and low SMR scores, but for the others no general tendency can be observed.

Figure 7
Fig. 7. Mean and standard deviation of the SMR bandpower for the low (green) and the high (red) group and for each subject. Subjects are sorted by their feedback performance (highest performance to the right).

Fig. 8 shows the grand average temporal evolution of the SMR amplitude during the trial in the two best CSP projection channels (one per class) for the two groups built according to pre-stimulus SMR band power (in panel Formula$a$: left-low in light blue, left-high in dark blue, right-low in pink and right-high in red) as well as the groups built according to the previous trial type (in panel Formula$b$: left-same in light green, left-diff in dark green, right-same in light orange and right-diff in dark orange).

Figure 8
Fig. 8. Grand average temporal evolution of the SMR amplitude in the best CSP projection channels (one per class). Time is represented on the x-axis: cue onset at Formula${t}=0$ and start of online feedback at Formula${t}=1000$ ms.

It can be observed that the SMR difference between the high and low groups in the post-stimulus interval is stronger for the CSP channel which synchronizes, while the CSP channel which desynchonizes is only affected by the group separation during the re-synchronization phase, which means that the SMR stays desynchronized for a longer period in the low group. This indicates that the main difference between the two groups appears in the ipsilateral hemisphere (left for left hand imagination, right for right hand imagination) where there is an ERS, instead of the contralateral one, where an ERD is present. It can also be seen that the separation between high and low trials starts already after 1 s from the end of the motor imagery task (5000 ms) and exhibits its peak within 1000 ms preceding the cue onset (which is because the separation is based on this pre-stimulus interval).

A smaller difference is present between diff and same trials in the post-stimulus interval: in the right hemisphere, a stronger ERS and ERD for the diff group is seen, while in the left hemisphere the diff group has a stronger but shorter ERD and the same group has a slightly stronger ERS. On the other hand, as expected [given the classification results in 4(b)], the two groups differ in the pre-stimulus interval, where the SMR is higher in the contralateral hemisphere compared to the previous trial (left, if the previous trial was right, i.e., independently from the present trial). This effect lasts until the ERD starts and can be assumed to originate from the rebound (ERS) following the previous motor imagery trial.

Fig. 9 represents the SMR amplitude scalp topographies for the four subgroups same-low, same-high, diff-low and diff-high and for each class left (panel a) and right (panel b). The SMR has been calculated and plotted for one pre-stimulus interval (Formula${-}1000$–0 ms) and five post-stimulus time intervals (0–500, 500–1000, 1000–1500, 1500–2000, and 2000–3000 ms).

Figure 9
Fig. 9. Grand average scalp topographies of SMR modulations for six time intervals. Cue onset is at Formula${t}=0$ ms and start of online feedback at Formula${t}=1000$ ms. Each panel shows the four trial group combinations for each class. (a) Left. (b) Right.
Figure 10
Fig. 10. Grand average scalp topographies of Formula$p$-values for the null hypothesis of zero biserial correlation between SMR amplitude and trial group. Cue onset is at Formula${t}=0$ ms and online feedback starts at Formula${t}=1000$ ms. The colorbar indicates the Formula$p$-value range for each panel. (a) High versus low. (b) Diff versus same.

Visual inspection of these topographies suggests that the main difference between the high and low trial groups is located in the ipsilateral hemisphere for each class. The difference between the diff and same trial groups is much weaker and is rather seen in the region corresponding to the ERD in the contralateral hemisphere. In more details, we observe that the ipsilateral SMR amplitude for the low group increases slightly during the early post-stimulus period (0–1000 ms) and then decreases with the contralateral SMR during the rest of the interval (1000–3000 ms). On the contrary, the ipsilateral SMR amplitude in the high group decreases through the trial, but still (and more importantly) remains at a higher level than the corresponding amplitude in the low group.

Complementary, we computed scalp topographies of the Formula$p$-values for the null hypothesis of zero biserial correlation between SMR amplitude and trial group (high/low and diff/same) membership in the grand average within each class (displayed in Fig. 10). These Formula$p$-value maps are shown only for the post-stimulus interval. Panel Formula$a$ displays the Formula$p$-values for high versus low trials, panel Formula$b$ displays the Formula$p$-values for diff versus same trials. The Formula$p$-value range for each panel is indicated by a color bar. The Formula$p$-value topographies in Fig. 10 further confirm the presence of an ipsilateral process as described above, since they indicate a highly significant Formula$({p} < 10^{-12})$ correlation between SMR amplitude and group membership in the ipsilateral hemisphere at the beginning of the trial, which decreases through the trial until the Formula$p$-value falls below 0.05. This is observed for both the diff and same trial groups and means that the SMR amplitude during motor imagery is higher over the ipsilateral motor cortex for the high, compared to the low trial group, regardless of which type of trial was previously performed. Furthermore, the lateralization of the effect is more strongly pronounced for the right class compared to the left class, where the contralateral activity remains higher for the high group, until about 1000–1500 ms and regardless of the previous trial.

Concerning the difference between the diff and same groups, we observe a higher correlation for the same group in the contralateral hand motor area Formula$({p} < 0.05)$, which represents the generally lower SMR amplitude for the diff group observed in Fig. 8(b). Moreover, a trend for higher correlation for the diff group in the ipsilateral area, representing higher SMR amplitudes for this group, can also be observed. Both these effects are seen rather early in the trial: 0–1000 ms for the contralateral effect and 0–500 ms for the ipsilateral one. Hence, no significant difference is observed during the later interval 1000–3000 ms, which corresponds to the main part of the pre-stimulus SMR effect on classification [see Fig. 4(a)].

C. Performance Predictability on Trial-Level and Subject-Level

Figure 11
Fig. 11. Predictability of performance on trial-level and subject-level. The x-axis displays each subject's feedback classification performance value. The y-axis shows the biserial correlation between the pre-stimulus SMR value and the highest classifier output value (over the time windows) for each trial.

In order to analyze how well the pre-stimulus SMR can predict the classification performance on a trial level, we calculated the biserial correlation between the pre-stimulus SMR pow score and the highest classifier output value (over the time windows) for each trial and each subject. The resulting values are plotted in Fig. 11 versus the feedback performance of each subject during the experiment. The biserial correlation values are rather high (ranging from 0.58 to 0.89), indicating a high predictability of the performance on a trial base. Furthermore, this predictability is significantly negatively correlated to feedback BCI performance (Formula$r = -0.61,$ Formula$p =0.001$). From this, we can infer that subjects who generally performed well depended less on the ongoing SMR level to produce well classifiable SMR patterns, than subjects who had more difficulties. However, because our analysis selected subjects which had at least 70% of classification accuracy, we can only compare medium to good BCI performers.

SECTION IV

DISCUSSION

A. Importance of Subject-Specific Parameters

In SMR-based BCI studies, SMR conventionally relates to the frequency band which is most useful for BCI control, i.e., displays the highest difference in desynchronization between the classes. Typically, it can involve the alpha, the beta or even both frequency bands and is highly user- [57] and localization-specific [58], [59] compared to the occipital alpha-rhythm. This was also demonstrated in our study, as the selected frequency bins showed two peaks: one in the alpha range (8–13 Hz) and one in the beta range (17–25 Hz). Still, most of the perception studies investigate the effect of pre-stimulus SMR within fixed frequency bands and the choice of these bands is not even congruent in the literature. This might explain the incongruent findings about the influence of ongoing SMR activity on the cortical processes during the subsequent task. The alpha and beta rhythmic components of the SMR differ from each other in topography and precise relationship to movement [38]. Localization studies restrict alpha-band components to somatosensory areas in the postcentral gyrus, while the beta components are limited to precentral areas involving the motor cortex [60]. Hence, between-user variability for frequency band and electrode selection could be explained by the use of different strategies for imagining motor acts, dependent on the involvement of sensation and/or motion feelings, along with inter-individual differences in brain topography. This emphasizes the need for user-specific parameter selections when defining the SMR.

B. Ipsilateral Hemisphere Idling Boosts Classification

This study demonstrates that trials with higher pre-stimulus SMR amplitude can be better classified than trials with lower SMR amplitude during this period. This effect is significant in the time interval corresponding to the ERD and suggests that the ongoing SMR amplitude has an effect on the ERD/ERS patterns induced by motor imagery.

Analyzing the SMR pattern evolution, it surprisingly turns out that the increase in classification for trials with higher pre-stimulus SMR level can not be attributed, as hypothesized, to a stronger ERD in the contralateral hemisphere, but to the persistence of a higher SMR level over the ipsilateral hemisphere. Based on evidence suggesting that an increase in SMR amplitude is preceded by muscle tone reduction [61], we expect this effect to be due to a deeper relaxation state of the sensorimotor system. In addition, [62] showed that high power in the alpha frequency band over the motor cortex makes it immune to external inputs by increasing its excitation threshold. Hence, we hypothesize that a deeper relaxation of the motor cortices leading to higher SMR amplitude will result in an ERD concentrated to the task-relevant (contralateral) cortical area accompanied by an ERS (idling) over the other areas. This hypothesis is in line with [63], who observed that the ipsilateral localized ERS often develops as the number of BCI feedback sessions increases and is further associated with an increase in the classification accuracy. This could be an effect of the subject getting used to the task and/or developing a skill and hence could be linked to both motor cortex and cognitive relaxation.

Due to the well documented attenuation of rhythmic activity in contralateral motor areas following unilateral upper-limb motor intentions [27], [29], research on SMR-based BCIs has mainly focused on the contralateral hemisphere. However, here, we provide evidence that the ipsilateral hemisphere also contains valuable information for decoding mental states. In line with our results, an electrocorticography (ECoG) study [64] reported cortical field potentials that were found to correlate with ipsilateral kinematics. Recent studies have also demonstrated that chronic stroke patients with limb paralysis can use noninvasively recorded ipsilateral SMR modulations to achieve BCI control [65] and that the ones who achieve a high level of recovery often display increased ipsilateral activation during movement [66]. Altogether, this supports the idea that alternative signals can be used for BCI control, when the regular signals cannot be recorded.

C. Stronger Pre-Stimulus Influence for Middle BCI Performers

The performance predictability on trial-level resulting in this study is higher than the subject-level performance predictability reported in [44] (Formula$r=0.58$ to Formula$r=0.89$ on trial-level against Formula$r=0.61$ on subject-level). Moreover, the subject-level predictability based on the pre-stimulus SMR is even higher than the one based on the SMR at rest in [44]. However, this is not so surprising, since our pre-stimulus analysis consider EEG recorded just before the task performance, while the SMR at rest was measured at least 1 h before and makes a comparison unfair. An interesting result is that the influence of the pre-stimulus SMR on the performance is significantly stronger for the half of the subjects with lowest feedback performance. Additionally, the trial group separability for these users was very small in comparison to the other half, thus a successful effect of the method described in this paper was not to be expected. All together, this result shows that a proper BCI or neurofeedback training, potentially in combination with newly developed machine learning online adaptation techniques [67], can successfully improve the performance of these users.

D. Beta Rebound Influences Classification at Trial Beginning

In parallel to the main effect the SMR amplitude of the previous trial has on classification performance, we observed that trials performed subsequently to a different trial type (i.e., left after right or right after left) can be better classified than trials performed after a similar trial type (i.e., left after left or right after right). This increase in classification begins early in the trial (even before the cue onset) and is accompanied by higher SMR amplitudes over the ipsilateral hand area, while a similar difference is observed over the contralateral area when the previous trial was similar. These observations can be explained by an influence of the post-movement beta rebound [68], [69] on task related SMR amplitude modulations over both hemispheres during the beginning of the trial. This rebound phenomenon is observed about 1 s after the end of the movement [69] or motor imagery [38] periods, following a somatotopic organization [70], [71]. Despite its name, post-movement beta rebound has been reported in both single and multiple frequency bands [38] (depending on the subject and the limb used, see [72]) and has its source in the motor cortex [68], which is compatible with the assumed generating region of the SMR. Alternatively, it is possible that switches in attention from one hand to the other might influence the classification performance very early in the trial. However, we would expect the participant to be quicker and better able to desynchronize accurately when he/she has to perform the same imagery twice in a row. However, this is not what the classification results show. A further hypothesis involves the generally lower SMR amplitude (along with the stronger desynchronization) present in trials following a trial of a different type. This difference may also contribute to the increase in classification. Yet, this hypothesis does not explain why the decrease in classification error is only significant until 1500 ms post-stimulus.

The influence of the beta rebound, which results in higher SMR in the ipsilateral hemisphere when the previous trial type was different, interacts with the high SMR in the ipsilateral hemisphere due to the high pre-stimulus SMR. However, this interaction is seen very early in the trial, as the beta rebound effect vanishes after around 500 ms from the beginning of the online feedback, while the main increase in the classification performance is in the middle of the trial. Nevertheless, this finding should be taken in consideration in future trial design, where a longer inter-stimulus interval would definitely help avoiding this drawback. Even better, a shorter trial can be considered, in order to hold a higher information transfer rate (ITR). A shorter trial length can be obtained adjusting the cue onset depending on the pre-stimulus SMR amplitude.

SECTION V

CONCLUSION

In this study, we demonstrate that ongoing SMR activity can influence the dynamics of the brain responses leading to changes in SMR-based BCI performance. More precisely, higher SMR amplitudes lead to better classifiable brain patterns. Interestingly, this increase in performance can be attributed to processes in the ipsilateral, rather than the contralateral hemisphere. Furthermore, we observed that medium BCI performers (around 70%–80% classification accuracy) benefit more from a higher SMR level before starting to perform motor imagery than good performers. We also found a small influence of the laterality of the preceding trial on the earlier part of this trial. Remarkably, this effect might as well be explained by an increase in ipsilateral SMR amplitude. Finally, our results support the idea that exploiting information about the ongoing SMR might be the key to boost performance in future SMR-BCI experiments. However, care should be taken for appropriate study design, in order to prevent trials to influence each other, as this is not compatible with the idea of single trial classification for BCI.

ACKNOWLEDGMENT

B. Blankertz proposed the topic of investigation. C. L. Maeder developed the simulation and carried out the data analysis, data classification, and visualization of the results. C. Sannelli and B. Blankertz assisted in the development of the methods and contributed to the interpretation of the results. S. Haufe contributed to the statistical analysis. C. L. Maeder produced the draft in all its sections. C. Sannelli revised the draft. All authors read and approved the manuscript. The authors declare that they have no competing interests.

Footnotes

This work was supported in part by grants from the Bundesministerium für Bildung und Forschung (BMBF), Fkz 01IB001A, 01GQ0850and in part by the German Science Foundation (DFG) under Contract MU 987/3-2.

C. L. Maeder and C. Sannelli are with the Berlin Institute of Technology, Machine Learning Laboratory, 10587 Berlin, Germany (e-mail: ceciliamaeder@gmail.com).

S. Haufe and B. Blankertz are with the Berlin Institute of Technology, Machine Learning Laboratory, 10587 Berlin, Germany, and also with the Bernstein Focus: Neurotechnology Berlin, 10587 Berlin, Germany.

References

No Data Available

Authors

No Photo Available

Cecilia L. Maeder

No Bio Available
No Photo Available

Claudia Sannelli

No Bio Available
No Photo Available

Stefan Haufe

No Bio Available
No Photo Available

Benjamin Blankertz

No Bio Available

Cited By

No Data Available

Keywords

Corrections

None

Multimedia

No Data Available
This paper appears in:
No Data Available
Issue Date:
No Data Available
On page(s):
No Data Available
ISSN:
None
INSPEC Accession Number:
None
Digital Object Identifier:
None
Date of Current Version:
No Data Available
Date of Original Publication:
No Data Available

Text Size