Multilevel Deep Reinforcement Learning for Secure Reservation-Based Electric Vehicle Charging via Differential Privacy and Energy Storage System

This paper presents a multilevel deep reinforcement learning (DRL) algorithm for a privacy-preserving charging of reserved individual electric vehicles (EVs) and the secure operation of a smart EV charging station (EVCS) installed with a solar photovoltaic system and energy storage system (ESS). At the first level, at each charging pole, the DRL agent enhances the data privacy of the reserved EV arrival/departure times at smart EVCSs using a discrete differential privacy method, ensuring nonoverlapping charging periods. At the second level, at all charging poles, multiple DRL agents cooperate to maximize the revenue of smart EVCSs, completely satisfying their charging demands. At the third level, an ESS DRL agent minimizes the operational energy cost of the smart EVCS while performing privacy-preserving energy management of the smart EVCS by flattening the net energy consumption to an economical target value via charging and discharging the ESS. The simulation results evaluated for smart EVCS with four charging poles confirm the effectiveness of the proposed three-level DRL algorithm in view of the privacy-preserving performance with varying privacy costs, increasing revenue, and decreasing operational energy cost of smart EVCS.