Evaluating Multilevel User Skill Expression in a Public, Unsupervised Wiki: A Case Study

Background: This case study examines how users of varied experience levels interact with an open-access content-management system (CMS) that lacks managed leadership. In particular, this case study compared how experienced and new users performed at a variety of tasks in a nonprofit community wiki to evaluate skill acquisition and possible skill loss. Research questions: (1) Do experienced users make more use of CMS features (templates, tags, hyperlinks) than new users, even without instruction? (2) Where do experienced and new users differ in successful task completion in the wiki? Situating the case: A number of content-management evaluations have utilized usability as a key method for evaluating user experience. In addition, Wikipedia has been a frequent target of investigation. This study looks to expand the crowd-sourced wikis evaluated in this manner. Methodology: This case study started with a series of interviews to determine how users expected to use the community wiki. After the interviews, 10 users (5 new and 5 experienced) were evaluated for this study, each performing 6 tasks. About the case: This study looked at the initial installation of a local community wiki system. The system is a CMS designed for use by municipal territories to create crowd-sourced wikis capable of preserving knowledge that would not traditionally fit in Wikipedia entries. Conclusions: Users who maintained sufficient interest in the wiki to become experienced wiki users developed a number of core skills even without organizational support, though new users demonstrated a steep skill deficit. However, new users actually demonstrated a greater capacity to highlight incompleteness of information within the wiki than experienced users in one key task.