An experimental evaluation of programming by contract

This paper presents an experiment designed to compare a contract-based programming method with a reference programming method based on exceptions. The purpose was to evaluate whether contracts would shorten the development time, improve work satisfaction and increase the quality of the resulting software program. The experiment was carried out in a project work course for students in the computer science program at Karlstad University. The students were to solve an assignment in groups of four within a period of ten weeks. Half of the groups used the contract-based method and the other half the exception based method. For statistical analysis we gathered data on time consumption and work satisfaction on daily report forms. The results show that there was a gain in the time spent on implementation of the assignment when the contract-based method is used, but show no significant difference in total time consumption. The results give a weak indication that work satisfaction was slightly higher in the groups using the contract-based method. The resulting programs were analyzed for a range of quality metrics, but no significant differences were found that were directly related to the methods.