Evaluating effectiveness of smartphone typing as an indicator of user emotion

In Affective Computing, different modalities, such as speech, facial expressions, physiological properties, smart-phone usage patterns, and their combinations, are applied to detect the affective states of a user. Keystroke analysis i.e. study of the typing behavior in desktop computer is found to be an effective modality for emotion detection because of its reliability, non-intrusiveness and low resource overhead. As smartphones proliferate, typing behavior on smartphone presents an equally powerful modality for emotion detection. It has the added advantage to run in-situ experiments with better coverage than the experiments using desktop computer keyboards. This work explores the efficacy of smartphone typing to detect multiple affective states. We use a qualitative and experimental approach to answer the question. We conduct an online survey among 120 participants to understand the typing habits in smartphones and collect feedback on multiple measurable parameters that affect their emotion while typing. The findings lead us to design and implement an Android based emotion detection system, TapSense, which can identify four different emotion states (happy, sad, stressed, relaxed) with an average accuracy (AUCROC) of 73% (maximum of 94%) based on typing features only. The analysis also reveals that among different features, typing speed is the most discriminative one.