Fully Automated Design Method Based on Reinforcement Learning and Surrogate Modeling for Antenna Array Decoupling

Modern electromagnetic (EM) device design generally relies on extensive iterative optimizations by designers using simulation software (e.g., CST), which is a very time-consuming and tedious process. To relieve human engineers and boost productivity, we proposed a machine learning (ML) framework to solve the problem of automated design for EM tasks. The proposed approach combines advanced reinforcement learning (RL) algorithms and deep neural networks (DNNs) in an attempt to simulate the decision-making process of human designers to realize automation learning. Specifically, the RL-based agent can interact with the EM design software without engaging human designers, allowing for automated design. Besides, the data accumulated during EM software simulation in the early design stage are reused as training data to build a DNN surrogate model to replace the time-consuming EM simulation and further accelerate the training of RL to achieve better optimization of EM design. Two types of antenna array decoupling including
$1\times $
2 and
$1\times $
4 arrays working at 3.5 GHz are used as test vehicles to validate the proposed method. The decoupling metasurfaces designed by the proposed fully automated method based on RL showed satisfactory results comparable to the results achievable by human designers. This indicates that the proposed method can be used to build powerful tools to boost the design efficiency of EM devices.