Bridging the Prediction-Decision Gap: Enhancing Model Deployment and Online Service Request Forecasting in Edge Inference Systems

In edge inference systems, efficient model deployment and accurate forecasting of online service requests are critical for maintaining optimal performance and resource utilization. This study introduces a novel approach that integrates predictive enhancement techniques to improve model deployment strategies and online service request forecasting. By addressing the existing gap between prediction and decision-making processes, our method enables more responsive and adaptive edge computing environments. Experimental results demonstrate significant improvements in both deployment efficiency and forecasting accuracy, highlighting the potential of our approach to advance the state-of-the-art in edge inference system management.