Optimizing Freshness of Machine Learning Model: A Timing (Multi-Armed) Bandit Approach

Maintaining the freshness of machine learning (ML) models, i.e., the time elapsed since their last update, is crucial for AI systems to deliver optimal performance. This is particularly relevant for resource-intensive models like Large Language Models, where frequent retraining or fine-tuning incurs significant costs. We address the challenge of jointly optimizing model freshness and update expense by carefully timing and tailoring ML model updates. We formulate this puzzle as a novel timing (multi-armed) bandit problem, where each "arm" represents a specific update timing with associated update costs and unknown, stochastic, model staleness cost. We develop an efficient online algorithm for the problem with provable sublinear regret. Our approach enables the dynamic allocation of resources towards retraining or fine-tuning, striking a balance between maintaining ML model freshness and minimizing update expenses.