Evaluating levels of automation in cooperative supervisory control: the mediator-associate experiment

The ISAM architecture provides an intelligent infrastructure for supporting cooperative work in real-time operations. ISAM represents activity, system, artifact, and information objects and utilizes a blackboard architecture to articulate current activity requirements in the context of current system state. It also supports two human operators performing cooperative supervisory control. This paper reports the results of an experimental evaluation of two versions of ISAM: ISAM as "mediator" in which a shared visualization of the blackboard is available to both operators, and ISAM as "associate" in which the ability for either operator to dynamically delegate activities to ISAM is supported. The experiment compares team performance in these conditions with respect to system performance measures (e.g., reaction time, errors) and coordination strategies (via detailed analyses of activity and verbalizations).