User Interface Defect Detection by Hesitation Analysis

Delays and errors are the frequent consequences of people having difficulty with a user interface. Such delays and errors can result in severe problems, particularly for mission-critical applications in which speed and accuracy are of the essence. User difficulty is often caused by interface-design defects that confuse or mislead users. Current techniques for isolating such defects are time-consuming and expensive, because they require human analysts to identify the points at which users experience difficulty; only then can diagnosis and repair of the defects take place. This paper presents an automated method for detecting instances of user difficulty based on identifying hesitations during system use. The method's accuracy was evaluated by comparing its judgments of user difficulty with ground truth generated by human analysts. The method's accuracy at a range of threshold parameter values is given; representative points include 92% of periods of user difficulty identified (with a 35% false-alarm rate); 86% (24% false-alarm rate); and 20% (3% false-alarm rate). Applications of the method to addressing interface defects are discussed