MetriStation: a tool for user-interface fault detection

Using extant methods of evaluation, user-interface fault detection is difficult, time consuming, error prone, often qualitative, and subject to evaluator bias. One of the more effective approaches, empirical evaluation, is data intensive, laboratory oriented, and requires up to 25 hours of skilled analyst time for every hour of testing time; hence it is seldom used. Preparing the evaluation is difficult; results can be inconsistent and hard to replicate. This paper presents MetriStation-a new, portable user-interface evaluation system that runs on an IBM-compatible desktop or notebook computer. MetriStation monitors user sessions, automatically capturing derailed data about user-system behavior including keystrokes, mouse clicks, mouse movements, video display context, user speech (think-aloud protocols), observer speech and critical incidents. Hesitation analysis is used for automatic location of potential user-interface defects. Data can be analyzed and presented in as much as three orders of magnitude less time than is conventionally possible; the uniformity of the data and analysis provide increased repeatability and accuracy over traditional evaluation methods. MetriStation's capabilities are demonstrated in a study of a simple interface.