Visual and Memory-based HCI Obstacles: Behaviour-based Detection and User Interface Adaptations Analysis

Human Computer Interaction (HCI) performance can be impaired by several HCI obstacles. Cognitive adaptive systems should dynamically detect such obstacles and compensate them with suitable User Interface (UI) adaptation. In this paper, we discuss the detection of two main HCI obstacles: memory-based and visual obstacles. A sequential model based on Long-Short Term Memory (LSTM) is suggested for such a detection of HCI obstacles. UI adaptations for both types of obstacles are discussed and analyzed. We investigate the classification performance on data from a user study with 17 participants. Furthermore, we also investigate the influence of different adaptation mechanisms on performance and subjective assessment. Results show advantages of the proposed sequential LSTM model: on the one hand, the LSTM outperforms the baseline random guess and also a baseline static model LDA in the detection of visual obstacles with 70.6% as an average accuracy. On the other hand, the evaluation of HCI sessions impeded by obstacles but supported with different UI adaptations shows that LSTM results well match the subjective assessment as a plausible detector of behaviour changes.