Abstraction as a predictor of difficulty in quizly problems

The Mobile Computer Science Principles curriculum collects data on embedded Quizly programming exercises, which are based on the App Inventor version of Blockly. We have recently started mining this data to determine whether student performance on the programming exercises matches our assumptions about the difficulty of the individual exercises. Various analytic techniques, such as linear regression, are used to identify those features that are most determinative of problem difficulty. Our analysis supports that the number of abstractions may be a useful predictor for the difficulty (defined for our data set as the average number of attempts) in solving Quizly exercises. However, there are other not so easily quantifiable factors that also affect a problem's difficulty.