A review of groupware evaluations

A better understanding of how groupware systems have been evaluated in the past can help to frame the discussion of what methods and techniques should be considered for future evaluations. We reviewed all papers from the ACM CSCW conference (1990-1998) that introduced or evaluated a groupware system. Forty-five papers were included in the review. The main findings are that almost one-third of the groupware systems were not evaluated in any formal way, that only about one-quarter of the articles included evaluations in a real-world setting, and that a wide variety of evaluation techniques are in use. Our main conclusions from the review are that more attention must be paid to evaluating groupware systems and that there is room for additional evaluation techniques that are simple and low in cost.