Predicting individual performance in student project teams

Due to the critical role of communication in project teams, capturing and analyzing developer design notes and conversations for use as performance predictors is becoming increasing important as software development processes become more asynchronous. Current prediction methods require human Subject Matter Experts (SME) to laboriously examine and rank user content along various categories such as participation and the information they express. SEREBRO is an integrated courseware tool that captures social and development artifacts automatically and provides real time rewards, in the form of badges and titles, indicating a user's progress towards predefined goals using a variety of automated assessment measures. The tool allows for instructor visualization, involvement, and feedback in the ongoing projects and provides avenues for the instructor to adapt or adjust project scope or individual role assignments based on past or current individual performance levels. This paper evaluates and compares the use of two automated SEREBRO measures with SME content-based analysis and work product grades as predictors of individual performance. Data is collected from undergraduate software engineering teams using SEREBRO, whose automated measures of content and contribution perform as well as SME ratings and grades to suggest individual performance can be predicted in real-time.