Learning Software Overtime Estimation From Experts‘ Annotations: A Greedy Cross-Validation-Based Machine Learning Approach

Effective overtime planning in software projects remains a critical challenge due to the complex interplay between project constraints and human-centric decision-making. This study introduces a novel machine learning-based approach to software overtime planning that learns from project managers’ annotations. Unique datasets were constructed from six real-world software projects, annotated by 20 experienced project managers, and refined using a clustering-based outlier removal technique to ensure consistency and reliability. The study evaluates multiple regression models and proposes a Greedy Halving Grid Search-optimized Random Forest Regressor (gHGS-EFR) to estimate project managers’ satisfaction with overtime allocation plans. Empirical results demonstrate that the baseline RFR model achieves strong predictive performance (average R2 = 0.81), outperforming linear and kernel-based models and rivaling neural networks and gradient-boosting methods. The greedy halving grid search optimization method significantly enhances model accuracy and scalability, reducing MAE and RMSE by 14.5% and 16.4%, respectively, and improving R2 to 0.89. Compared to traditional hyperparameter tuning methods, gHGS offers superior computational efficiency, reducing wall-clock time by up to 70%. Moreover, the gHGS-RFR model maintains stable performance across increasing project complexity, highlighting its robustness and practical applicability. The SHAP interpretability analysis identified which project features most influence satisfaction scores and how their characteristics affect overtime estimation, offering actionable insights for project managers. This work represents the first application of machine learning to software overtime planning using expert-annotated data, bridging the gap between algorithmic planning and human judgment.