Structural Optimization for Asymmetrical Inline Topology Filter With Transmission Zeros Using Goal-Oriented Reinforcement Learning

This paper presents a novel structural optimization approach for an asymmetrical inline topology with transmission zeros, known as Extracted Pole Unit (EPU) filter using a goal-oriented reinforcement learning method, specifically a hybrid of Soft Actor Critic (SAC) and Hindsight Experience Replay (HER). The recent popularity of reinforcement learning (RL) algorithms for optimizing cavity bandpass filters (BPFs) has led to several limitations. RL algorithms are inherently sample-inefficient, leading to prolonged model training times. A substantial number of training samples are required to achieve accurate results for a complex design using RL models. Additionally, most objective functions used for fitness calculations do not account for predistortion constants, which are crucial in synthesizing asymmetrical inline topologies with three transmission zeros as demonstrated in this work. To address these challenges, the proposed method incorporates predistortion-modified poles and transmission zeros within a feature-assisted objective function for use in the optimization process. Subsequently, a hybrid of SAC and HER is adopted as the optimization algorithm to leverage its improved sample efficiency by encouraging learning from diverse optimization scenarios and outcomes. The proposed method can optimize the self-couplings of the EPU filter in fewer optimization steps, showcasing enhanced training convergence speed and design accuracy.