Adaptive Auto-Scaling of Distributed Databases on Cloud Platforms via Reinforcement Learning-based Resource Optimization

With the integration and development of cloud platforms and distributed database technologies, how to achieve efficient resource scheduling and dynamic elastic scaling in a multi-tenant, high-concurrency and resource heterogeneous environment has become an important challenge. This study proposes a resource optimization algorithm that integrates deep reinforcement learning and adaptive elastic scaling mechanism. The application scenario is to improve the response efficiency and resource utilization of cloud platform distributed database systems. At the algorithm design level, the study first models the resource scheduling problem as a Markov decision process (MDP) with a continuous state space and action space. At the same time, a deep deterministic policy gradient algorithm is introduced to build an intelligent agent. This operation can realize the self-learning of the CPU allocation strategy of the database node. Secondly, a parameter tuning model based on Bayesian optimization is constructed. This can support the dynamic adjustment of the total amount of resources to realize the automatic increase and decrease of nodes and service level guarantee. Finally, a data integrity coordination mechanism is introduced. This can improve the decoding success rate of the data transmission process while supporting resource scheduling and elastic scaling. To verify the effectiveness of the proposed method, this study built a simulation test platform in a virtualized cloud platform environment. At the same time, the Apache Cassandra distributed database was deployed, in conjunction with the Kubernetes container scheduling environment and the JMeter+Locust business generator. The proposed method maintained the shortest response delay (minimum 192ms) under peak, stable and burst load conditions, and the CPU and memory utilization rates reached 82.5% and 77.8% respectively. The proposed method is significantly superior to traditional algorithms in terms of the system performance improvement, resource optimization and data integrity assurance.