Gaps Between Continuous Measurement Methods: A Longitudinal Study of Perceived Usability

The available continuous measurement designs for longitudinal research on perceived usability include immediate and retrospective measurement methods. The present study was conducted to examine the gap between different continuous measurement designs. Thirty users completed a longitudinal study on typesetting software. We used two measurement methods to design three stages of testing. Each stage consists of one immediate measurement and two retrospective measurements. A long-term retrospective test was also conducted and SUS questionnaire measurements were gathered. We observed significant differences in perceived usability between participants with varying quantities of event experiences (i.e. the number of times they used the system). Within short periods of time (3 days), there may be significant differences between retrospective and immediate assessments—especially when users make negative evaluations. In the 3-day period, the user’s perceived usability retrospective evaluation is also based on the most recent experience as opposed to the most unpleasant. The recency effect is lost, however, over a long-term retrospective assessment (1 month later). This study is significant for two main reasons: (i) its unique methodology and (ii) its unprecedented level of consistency between measured results and actual user experiences. We hope that the results presented here provide a workable basis for continuous measurement design in future longitudinal studies. We found that in order to secure accurate measurements, it is necessary to set limits on the measurement time and the number of uses (events).