Model-Free Deep Reinforcement Learning-based Current Control for the Dual-Purpose dc-dc/ac Power Converter

The recently proposed three-phase dual-purpose dc-ac/dc power converter (PC) as the new member of the universal converter family shows high potential for facilitating the integration of renewable energy sources (RESs) and energy storage systems (ESS) in any type of grid (dc or ac), aiming at high performance and versatility with minimal redundancy or modification. The proposed dual-purpose PC requires flexible, adaptive, reliable and scalable control strategies that align with the goal of providing a standardized solution for the power conversion stages in the coexisting ac and dc microgrids (MGs). In this paper, a model-free deep reinforcement learning (DRL)-based current control methodology is proposed for that dual-purpose PC. The non-linear and model-free nature of the proposed approach reduces the system sensitivity to parameter variations, increasing the system robustness and reliability; and can potentially reduce the development and operational costs since a detailed modeling and an extensive parameter tuning are avoided. Two DRL agents were trained by means of the twin delayed deep deterministic (TD3) policy gradient algorithm for both dc-ac and dc-dc modes of operation. Simulation results are provided to demonstrate the performance of the proposed control strategy.