Emotion recognition in the manual interaction with graphical user interfaces

We introduce a novel approach to human emotion recognition, based on manual computer interaction. The presented methods rely on conventional graphical input devices. Firstly, a standard mouse as used on desktop PCs, and, secondly, the interaction with touch-screens or -pads as in public information terminals, palm-top devices or tablet PCs is considered. Additionally, the gain of the integration of touch pressure information is evaluated. Four discrete emotional states are classified: irritation, annoyance, reflectiveness, and neutral affect, for use in initiative tutoring, error clarification, Internet customer personalization, and others. The optimal feature-set is discussed and ranked according to a linear discriminant analysis. A working system, using support vector machines for the classification, is tested in real-life scenarios. A performance of up to 83.2% correct assignment clearly indicates that user emotion recognition is possible without special hardware in any standard graphical user environment, independent of the underlying application.