Evaluation of Performance Metrics for Users of Brain Computer Interfaces during Motor Imagery

It is well-known that proper training is indispensable for users of Brain-Computer Interfaces (BCI) to acquire the required skills to control the system, particularly for BCI based on motor imagery (MI-BCI). However, in order to assess the effectivity of the training procedure, it is necessary to evaluate separately the classification algorithm and the BCI user skills. Recently, new performance metrics to quantify MIBCI skills regardless the classification algorithm, by defining class distinctiveness metrics and trial stability metrics based on Riemannian distance, was proposed. In this study, we calculated such metrics for two balanced datasets containing EEG recorded with slightly different protocols from 30 age-, gender- and education-matched subjects during MI of right-hand and left-hand movements. The protocols revealed almost no difference regarding the class distinctiveness metrics, but showed great difference when it comes to the trial stability metrics. The results of this analysis can guide the improvement of protocols for BCI based on the motor imagery paradigm.