Improving control panel consistency of Wizard-of-Oz design and evaluation studies

This paper investigates how a Wizard of Oz (WoZ) control panel could be developed to improve `between-subject' consistency. To achieve this we conducted a comparative study of two control panels. Both control panels were used by the experimenter to ostensibly facilitate the design and evaluation of a novel domestic planning application allowing members of a family to coordinate a range of social arrangements and tasks. Based on video analysis and semi-formal interviews, the control panels as reliable design and evaluation tools were assessed. Results suggested that the component-separated control panel could obviously improve operational effectiveness thus enhancing system consistency.