Cost Function: A Cross-Domain Review of Design, Optimization, and Application in Engineering and Applied Sciences

Cost functions serve as the backbone of machine learning models, guiding optimization processes and significantly influencing predictive accuracy and model generalization. This review presents a comprehensive analysis of cost function design and application across a broad spectrum of engineering and applied science domains, including manufacturing, structural mechanics, computational design, biomechanics, reinforcement learning, and construction management. By synthesizing insights from 20 recent studies, the paper explores how cost functions are formulated, adapted, and optimized to suit complex, domain-specific challenges— ranging from estimating machining costs using explainable AI to learning physical laws via energy-based deep learning methods. Special attention is given to the integration of traditional engineering knowledge into data-driven models, the role of interpretability in cost function design, and the emergence of hybrid frameworks combining physical simulation with neural architectures. This work highlights both the diversity and convergence of cost function methodologies, identifies trends in scalable and interpretable modeling, and outlines research directions for the development of adaptive, data-efficient, and context-aware cost functions in engineering applications.