Digital Twin-Based Reinforcement Learning for Energy Exchange Among Electric Vehicles and Base Stations in a Disaster-Affected Region

The cellular base stations (BSs) have backup batteries to maintain uninterrupted power supply. Recent studies have shown that a backup battery may have some spare energy to act as a flexible resource. Similarly, electric vehicles (EVs) are also capable to give surplus energy stored in their batteries to other consumers or back to the grid. Therefore, both BSs and EVs can share energy among themselves through Telecom-to-Vehicle (T2V) and Vehicle-to-Telecom (V2T) exchange. However, the energy exchange is challenging in a disaster-affected region due to connectivity failures, power disruption and damaged routes. This paper proposes an energy exchange solution among BSs and EVs in a post disaster situation. We propose a digital-twin (DT) based solution which utilizes Artificial Intelligence (AI) algorithms to estimate energy consumption of BSs and EVs and identifies their role as energy buyers or sellers. It also models power disruption and disaster-affected blocked routes as Markov processes with parameters derived from real historic data of floods. Then, a reinforcement learning (RL) algorithm is proposed to match BSs and EVs which can feasibly take part in either T2V or V2T exchange. Performance of the proposed solution is compared with independent RL without DT and assisted by federated learning. Simulations show that energy exchanged by RL algorithm doubles with the utilization of DT.