Do Estimators Learn? On the Effect of a Positively Skewed Distribution of Effort Data on Software Portfolio Productivity

We study whether an assumed positively skewed distribution ofeffort data prevents software estimators to learn over time; leadingto increasing differences between planned and actual effort and adeteriorating (worsening) trend on productivity. We analyze dataof 25 software releases of one application, collected over a periodof six years in a public sector institution in The Netherlands. Westatistically test for distribution, trend on differences betweenplanned versus actual effort over time, and productivity of softwareportfolios. The key contributions of this paper are that we show thata proposed assumption that assumes any relation between apositively skewed distribution of effort data and a deterioratingproductivity is not applicable to the subject dataset. We find thatthe effort data is to be characterized as positively skeweddistributed, and we do see a shift over time from under-estimationto over-estimation. We do not find evidence for a deterioratingproductivity; on the contrary productivity improves over time,indicating that estimators in the subject organization did learn.