Structural Charging and Replenishment Policies for Battery Swapping Charging System Operation Under Uncertainty

We study the joint battery charging and replenishment scheduling of a battery swapping charging system (BSCS) considering random electric vehicle (EV) arrivals, renewable generation, and electricity prices. We formulate the problem as a Markov decision process with an objective to minimize the expected sum of the operation cost (battery charging and replenishment cost) and the waiting cost of EV customers. The joint scheduling problem is challenging due to the stochasticity in EV arrivals, renewable generation, and electricity prices, as well as the curse of dimensionality in the system state and action spaces. To reduce the dimension of the action space, we propose to integrate structural properties into BSCS operation, i.e., the threshold-charging (TC) and least demand first (LDF) structures into the charging policy, and the
$(s, S)$
structure into the replenishment policy (when the number of fully-charged batteries at a battery swapping station is below
$s$
, the inventory is replenished to a higher threshold
$S$
). Numerical experiments on real-world data show that the proposed SAC+TC+
$(s, S)$
approach saves 7.16%-78.61% and 6.53%-93.73% of total average cost resulting from various structural charging and replenishment policies and the vanilla soft actor-critic (SAC) algorithm under different settings.