A Comparative Study on COSMIC FP Approximation with Deep Learning and Conventional Machine Learning

CONTEXT: Functional size measurement (FSM) is a basis of project planning. FSM methods usually need experts and well-specified requirements. To relax these constraints, automated size approximation models have been developed. A recent study proposed a deep learning based model named DEEP-COSMIC-UC for COSMIC FP approximation based on use cases. Performance comparison in that study did not consider conventional machine learning techniques and contextualized word embeddings. OBJECTIVE: To evaluate the effectiveness of the deep learning based COSMIC approximation. METHOD: Empirical experiments were conducted with actual project data. Random Forests and Support Vector Regression were used as representative conventional machine learning techniques. BERT-based model was used as a representative contextualized word embedding model. RESULTS: No clear superiority of DEEP-COSMIC-UC was observed. CONCLUSION: Simple off-the-shelf models are enough at least for the dataset we used.