Investigating Ensemble Learning and Classifier Generalization in a Hybrid, Passive Brain-Computer Interface for Assessing Cognitive Workload

Hybrid, passive brain-computer (h/pBCI) interfaces have received much attention in regards to measuring various mental states. A high classification rate of operator workload state is necessary in order to be able to enhance operator performance. Physiological measures have been used with machine learning algorithms to classify workload state, however, these measures are hypothesized to suffer from inherent nonstationarity. To attain a more generalizable classifier, a prior solution has been to use a multi-day learning paradigm to train classifier models. In earlier work, we have shown that increasing the number of unique data sessions used to form a learning set can improve the accuracy of classifying mental workload where improved generalizability is partly attributable to the multi-day paradigm. To further investigate methods that produce more generalizable classifiers, we look to ensemble learning. Here we implement ensemble learning to increase accuracies, reduce variance, and leverage theoretical performance of the ensemble as compared to observed to make inference about generalization. An adaptive boosting method (AdaBoost) is used to train a "base learning algorithm" multiple times, adaptively adjusting to errors and forming a vote out of the resulting hypotheses using three different base learning algorithms: an artificial neural network (ANN), a support vector machine (SVM), and linear discriminant analysis (LDA). We observed that the ensemble converged on theoretical performance with respect to error and variance only when the training sets were formed using the multi-day paradigm. These results indicate that ensemble learning approaches can be used in examples of pBCI such as those designed here, but they are also affected by theorized nonstationarity in physiological response. The observation of ensemble convergence on theoretical performance may be used to make inference about generalizability when simple accuracy of detection can be misleading.