Factors determining the difficulty of practice problems in a computer language textbook

We studied the factors which determine the difficulty of practice problems in a computer language textbook. The extraction of these factors from such complex problems requires considerable effort, and after extraction these factors are difficult to analyze because of their large number. An additional difficulty is that these factors cannot include the full range of human cognitive characteristics involved in problem solving. To overcome these problems, we took the following two steps. First, we undertook an analysis with formal factors, namely the knowledge and inferences required to solve a problem, and estimated the degree of difficulty. Second we conducted an error analysis on specific problems which included some deviation from the formal factors, and extracted the factors required for a special cognitive load. Our model was then made more accurate by including these factors among those which were found to be significant. In this way, we estimated the difficulty of practice problems quantitatively, and showed the effectiveness of the proposed method by means of an experiment using an actual C language textbook.