Hitting the wall: errors in developing and debugging a "simple" spreadsheet model

Undergraduate MIS business students developed and debugged a spreadsheet model from a word problem. This model consisted of a bid to build a wall. The problem was designed to be relatively simple and domain-free to address the concern that past spreadsheet experiments may have used problems that were too difficult or that required domain knowledge that subjects did not have. During the development phase, 72 subjects created the spreadsheet model. Even with this rather simple problem, 38% of the models contained an error. This high number of incorrect spreadsheets was not due to subjects making many errors. They only made 0.4 errors per spreadsheet. In addition, their cell error rate (CER) was only 1.7%, meaning that only 1.7% of their cells contained errors. Unfortunately, spreadsheets tend to have long cascades of cells leading to the bottom line. This means that even tiny cell error rates will multiply into high rates of bottom-line errors. In a debugging phase, subjects tried to debug their own models. Of 19 subjects with incorrect models who did the debugging part of the experiment, only three (16%) found and corrected their errors. So even with a relatively simple model, development and debugging were problematic. This is a fewer rate of finding errors than (Galletta et al., 1993; 1996) found when subjects debugged models created by the experimenter. This may mean that people are not as good at debugging their own models as they are at debugging models created by others.