Parametric Modeling of EM Behavior of Microwave Components Using Combined Neural Networks and Hybrid-Based Transfer Functions

Neuro-transfer function (neuro-TF) approaches have become more and more popular in parametric modeling for electromagnetic (EM) behavior of microwave components. Existing pole-residue-based neuro-TF approach has better capability of dealing with high-order problem than the rational-based neuro-TF approach, but has the discontinuity issue and the associated non-smoothness issue of the poles/residues when the geometrical variations become large while the rational-based neuro-TF approach does not have. This paper addresses this situation and presents a novel hybrid-based neuro-TF technique which systematically combines both pole-residue and rational formats of the transfer functions. Starting with the pole-residue-based transfer functions, we propose a novel technique to automatically identify the poles/residues that are smooth-continuous and the poles/residues that have the discontinuity and non-smoothness issues. The proposed technique converts the poles/residues that have those issues into the coefﬁcients of the rational-based transfer function to solve the discontinuity and non-smoothness issues in the existing pole-residue-based neuro-TF approach. The proposed technique remains the smooth-continuous poles/residues in the pole-residue format of the transfer function to maintain the capability of handling high-order problem. Compared with the existing neuro-TF modeling methods, the proposed technique can obtain better accuracy in challenging applications of large geometrical variations and high order. The proposed technique is illustrated by two examples of parametric modeling of microwave components.


I. INTRODUCTION
Artificial neural network (ANN) has been recognized as a powerful tool for parametric modeling of electromagnetic (EM) behaviors in microwave area [1]- [5]. EM design optimization can be expensive because it requires repetitive EM simulations due to adjustments of the values of geometrical parameters. Through a systematic computer-based training process, ANN can be used to learn the relationship between EM behaviors and geometrical parameters. After training, the developed parametric model can be used to The associate editor coordinating the review of this manuscript and approving it for publication was Chan Hwang See . provide accurate and fast prediction of the EM behavior of microwave components, and can be subsequently implemented in high-level circuit and system design [6].
Another parametric modeling approach is the knowledgebased approach. The knowledge-based neural network (KBNN) is a recognized knowledge-based approach for parametric modeling. In the KBNN approach, KBNN models are developed using neural networks combined with knowledge models such as analytical formulas [7], empirical functions [8], or equivalent circuit models [9], [10]. The knowledge models can help speed up model development and enhance the capability for learning and generalization of the KBNN models [11]. Space mapping [12]- [19] is also a recognized knowledge-based approach for parametric modeling. Research endeavors on space mapping have focused on several areas, such as implicit space mapping [13], output space mapping [14], [15], neural space mapping [16], [17], etc.
Another parametric modeling approach is the neurotransfer function (neuro-TF) approach [20], [21], which combines neural networks and transfer functions. In this approach, EM responses of passive components versus frequency are represented by the transfer functions. This approach can be employed even if accurate knowledge models are not available. The pole-residue-based neuro-TF approach [22] is one of the commonly used neuro-TF approach. This approach can work well when the geometrical variations are small. However, as the geometrical variations become large, some poles/residues are no longer complex w.r.t. geometrical parameters, resulting in the discontinuity issue. Along with the discontinuity issue in the poles/residues, the smoothness of the neighboring poles/residues w.r.t. the geometrical parameters are also affected, resulting in the non-smoothness issue. These issues cause the difficulty in training for the pole-residuebased neuro-TF model. For this reason, the effectiveness of the approach is compromised when geometrical variations become large. Another neuro-TF approach is the rationalbased neuro-TF approach [23], which does not have the discontinuity issue existed in the poles/residues. However, as the order of the transfer function increases and/or the geometrical variations become large, the transfer function is more sensitive to the coefficients, resulting in the difficulty in training for the rational-based neuro-TF model. The sensitivities of the transfer function responses w.r.t. the coefficients and the poles/residues are illustrated in [22]. For this reason, the order of the transfer function is often limited to a small number, compromising effectiveness of the rational-based neuro-TF approach to the high order applications.
In this paper, a novel parametric modeling technique is proposed to develop a combined neural network and hybridbased transfer function (hybrid-based neuro-TF) model of EM behavior of microwave components. The proposed technique systematically combines both pole-residue and rational formats of the transfer functions. Starting with the poleresidue-based transfer functions, the proposed technique automatically identify the poles/residues that are smoothcontinuous and the poles/residues that have the discontinuity and non-smoothness issues. The proposed technique converts the poles/residues that have those issues into the coefficients of the rational-based transfer function to effectively address the discontinuity and non-smoothness issues in the existing pole-residue-based neuro-TF approach. The proposed technique remains the smooth-continuous poles/residues in the pole-residue format of the transfer function to maintain the capability of handling high-order problem. Compared with the existing rational-based neuro-TF modeling method and the existing pole-residue-based neuro-TF modeling method, the proposed technique can obtain better accuracy in challenging applications of large geometrical variations and high order, addressing the discontinuity and non-smoothness issues. After the proposed modeling process is finished, the developed model can provide accurate and fast prediction of the EM behavior of microwave components and can be subsequently used in high-level circuit and system design.
The organization of this paper is as follows. In Section II, we described the discontinuity issue and the non-smoothness issue in the poles/residues w.r.t. geometrical parameters. In Section III, we described the proposed technique for developing a hybrid-based neuro-TF model in details.
In Section IV, we demonstrated the parametric modeling process by two EM examples, confirming the effectiveness of our proposed hybrid-based neuro-TF modeling technique.

II. THE ISSUES OF DISCONTINUITY AND ACCOMPANIED NON-SMOOTHNESS IN POLES AND RESIDUES W.R.T. GEOMETRICAL PARAMETERS
In the existing pole-residue-based neuro-TF modeling approach [22], a parameter extraction process is required to extract the poles/residues in the transfer function for each geometrical parameter sample. All the extracted poles/residues are required to be changed continuously in terms of the types of their values as the geometrical parameters change. The method can work well when the geometrical variations are small. It is because that all the poles/residues vary within complex values, which are continuous as the geometrical parameters change. However, as the geometrical variations become large, not all the poles/residues vary within complex values as the geometrical parameters change. In this case, the discontinuity issue occurs in the poles/residues and becomes a major issue for parametric modeling. More specifically, some poles/residues vary within complex values, while other poles/residues vary between real values and complex values, as the geometrical parameters change. The discontinuity issue is also the cause for the high nonlinearity of pole/residues from sample to sample, resulting in the difficulty of the training for the pole-residue-based neuro-TF model. For this reason, the effectiveness of the method is compromised when the discontinuity issue exists in the poles/residues as the geometrical variations become large. Along with the discontinuity issue in the poles/residues, the smoothness of the neighboring poles/residues w.r.t. the geometrical parameters are also affected. Although some poles/residues do not have the discontinuity issue as other poles/residues, some of these continuous poles/residues are not smooth w.r.t. the geometrical parameters, leading to the non-smoothness issue. This issue is also the cause for the high nonlinearity of pole/residues from sample to sample, resulting in the difficulty of the training for the pole-residue-based neuro-TF model. This is another reason why the effectiveness of the method is compromised when the discontinuity issue exists in the poles/residues as the geometrical variations become large. Rational-based neuro-TF modeling approach is one of the methods that does not have the discontinuity issue existed in the pole-residue-based neuro-TF approach. In the existing rational-based neuro-TF modeling approach [23], the poles/residues are reformatted into the coefficients of the numerators and the denominators of the rational-based transfer function. These coefficients always vary in real values as the the geometrical parameters change. In other words, the coefficients are continuous and does not have the discontinuity issue. However, as the order of the transfer function increases, the transfer function is more sensitive to the coefficients, resulting in the difficulty of the training for the rational-based neuro-TF model. For this reason, the effectiveness of the method is compromised when the order of the transfer function is high.
The focus of this paper is on developing a new and effective parametric modeling technique, addressing the discontinuity issue and the accompanied non-smoothness issue. In the subsequent section, we introduce a novel hybridbased neuro-TF modeling technique which systematically combines both pole-residue and rational formats of the transfer functions in order to overcome those issues in the poles/residues. The main idea is that we convert the poles/residues that have the issues into the coefficients of the rational-based transfer function to effectively address the discontinuity and non-smoothness issues and we remain the smooth-continuous poles/residues in the pole-residue format of the transfer function to maintain the capability of handling high-order problem. In this way, the discontinuity and non-smoothness issues can be solved by the proposed hybrid-based neuro-TF modeling technique and high-order modeling capability can be maintained by the overall model.

III. PROPOSED HYBRID-BASED NEURO-TF METHOD FOR PARAMETRIC MODEL DEVELOPMENT A. FORMULATION OF THE HYBRID-BASED NEURO-TF MODEL
The structure of the proposed hybrid-based neuro-TF model is illustrated in Fig. 1. Let x be the model input, representing the geometrical variables. Let y be the model output, representing the EM responses (e.g., S-parameters). The proposed model consists of the pole-residue-based transfer function, the rational-based transfer function, and four neural networks. The structure of the proposed model is a little bit more complicated than the existing pole-residue-based neuro-TF or the existing rational-based neuro-TF. The outputs of the model are the EM behaviors e.g., S-parameters, of the microwave components. The inputs of the model are the geometrical parameters of the microwave components and frequency. As the values of geometrical parameters change, some of the pole/residues may vary between real and complex values, resulting in the discontinuity issue. Along with the discontinuity issue in the poles/residues, the smoothness of the neighboring poles/residues w.r.t. the geometrical parameters are also affected, leading to the non-smoothness issue. Here, we define the continuous pole/residue to be the one whose value is always complex or real as the values of the geometrical parameters change. We define the discontinuous pole/residue to be the one whose value varies between real and complex as the value of the geometrical parameters change. Among the continuous poles/residues, there may exist one or more poles/residues whose values do not change smoothly as the values of the geometrical parameters change. The non-smoothness issue happens in these continuous poles/residues whose smoothness w.r.t. the geometrical parameters does not satisfy the smoothness requirement.
Let these poles/residues be called the non-smooth-continuous poles/residues. The rest of the continuous poles/residues satisfy the smoothness requirement. Let the remaining smooth and continuous poles/residues be called the smoothcontinuous poles/residues. We propose to maintain the smooth-continuous poles/ residues to formulate the pole-residue-based transfer function, while to re-formulate the discontinuous poles/residues and the non-smooth-continuous poles/residues together into rational-based transfer function. The pole-residue-based transfer function and the rational-format-based transfer function are combined together to formulate the proposed hybridbased transfer function. By such systematical combination of the two transfer functions, the discontinuity and non-smoothness issues can be effectively addressed by the rational-based transfer function with a relatively low order and the capability of handling high-order problem can be maintained by the pole-residue-based transfer function without the issues. In this way, the discontinuity and non-smoothness issues can be solved by the proposed hybridbased neuro-TF modeling technique and high-order modeling capability can be maintained by the overall model. The parameters in the proposed hybrid-based transfer function are defined as the hybrid parameters, containing the poles/residues of the pole-residue-based transfer function and coefficients in the numerator/denominator of the rationalbased transfer function. Since the relationship between the hybrid parameters and the geometrical parameters is nonlinear and unknown, we propose to use neural networks to learn and represent this nonlinear relationship.
Let N 1 and N 2 represent the orders of the pole-residuebased transfer function and the rational-based transfer function, respectively. Let N represent the total order of the hybrid-based transfer function, which equals the sum of the N 1 and N 2 , i.e., N = N 1 + N 2 . In the propose model, four different neural networks are used to learn and represent the nonlinear and unknown relationship between the hybrid parameters and the geometrical parameters. More specifically, these neural networks are used to learn and represent the nonlinear and unknown relationship between the poles/residues of the pole-residue-based transfer function and the geometrical parameters, and the nonlinear and unknown relationship between the coefficients of the rational-based transfer function and the geometrical parameters. Let p i represent the ith output of the neural network for poles corresponding to geometrical variables and neural network weights w p . Let r i represent the ith output of the neural network for residues corresponding to geometrical variables and neural network weights w r . Let a i represent the ith output of the neural network for the coefficients in the numerator of the rationalbased transfer function corresponding to geometrical variables and neural network weights w a . Let b i represent the ith output of the neural network for the coefficients in the denominator of the rational-based transfer function corresponding to geometrical variables and neural network weights w b . Let w represent a vector containing all the weights in the proposed hybrid-based neuro-TF model, defined as Let x be a vector containing the geometrical variables, representing the inputs of the proposed hybrid-based neuro-TF model. Let y represent the output frequency response, e.g., S-parameter of the proposed model. For convenience of representation, we define p and r to be vectors containing the smooth-continuous poles and residues, respectively, i.e., p = [p 1 p 2 · · · p N 1 ] T and r = [r 1 r 2 · · · r N 1 ] T . We define a and b to be vectors containing coefficients in the numerator and denominator of the rational-based transfer function, respectively, i.e., a = [a 1 a 2 · · · a N 2 ] T and As is known to all, the pole-residuebased transfer function and the rational-based transfer function are interconvertible to represent the frequency response in general. Instead of using these two classical transfer functions, we propose to convert some of the pole-residue terms from the original pole-residue-based transfer function into the rational-based transfer function in order to formulate the hybrid-based neuro-TF model. The output y of the hybridbased neuro-TF model can be formulated as where s represents the frequency in Laplace domain.

B. PROPOSED DERIVATIVE CALCULATION FOR HYBRID-BASED NEURO-TF MODEL DEVELOPMENT
In this subsection, we derive the derivative formulation for training of the proposed hybrid-based neuro-TF model. Let ∂y/∂p i and ∂y/∂r i represent the derivative of the output y w.r.t. the ith pole p i and the ith residue r i of the pole-residuebased transfer function, respectively. The derivatives ∂y/∂p i and ∂y/∂r i are formulated as and The derivative vectors of the output y w.r.t. p and r are defined as ∂y/∂p and ∂y/∂r which contain the derivatives ∂y/∂p i and ∂y/∂r i , respectively, defined as and ∂y ∂r = ∂y ∂r 1 ∂y ∂r 2 · · · ∂y ∂r N 1 T (6) VOLUME 8, 2020 Let α and β represent the values of the numerator and the denominator of the rational-based transfer function, respectively, expressed as and The derivatives of the output y w.r.t. the ith coefficient a i in the numerator and the ith coefficient b i in the denominator of the rational-based transfer function are defined as ∂y/∂a i and ∂y/∂b i , respectively, formulated as and The derivative vectors of output y w.r.t. a and b are defined as ∂y/∂a and ∂y/∂b which contain the derivatives ∂y/∂a i and ∂y/∂b i , respectively, defined as and ∂y ∂b = Let s represent a vector containing different orders of frequency s, defined as, The derivative vectors ∂y/∂a and ∂y/∂b can be formulated as and The derivatives of neural network outputs p, r, a, and b w.r.t. neural network weighting parameters w are defined as P, R, A, and B, respectively. The derivatives P, R, A, and B can be calculated using the algorithms in [5], where P(x, w) = ∂p/∂w, R(x, w) = ∂r/∂w, A(x, w) = ∂a/∂w, and B(x, w) = ∂b/∂w. Let ∂y/∂w be defined as the derivatives of model output y w.r.t. weighting parameters w. Substituting (3)−(15), the derivatives ∂y/∂w are formulated as The derivatives formulated in this subsection are used for training of the proposed hybrid-based neuro-TF model. In order to accurately train the proposed model, the initial values for the neural network weights are very important. To obtain good initial values for the weights, preliminary training of the neural networks is necessary before training the overall model [22]. To perform the preliminary training of the neural networks, the data for the hybrid parameters need to be extracted from the EM behaviors (e.g., S-parameters). In the next subsection, we describe the proposed hybrid parameter extraction technique.

C. PROPOSED HYBRID PARAMETER EXTRACTION TECHNIQUE
The proposed hybrid parameter extraction technique begins with samples of EM data d k , e.g., S-parameters, for different values of geometrical parameters x k , where the subscript k represents the index indicating the kth sample of geometrical parameters, i.e., k ∈ T r = {1, 2, . . . , n s }, where T r is the index set of training samples of geometrical parameters and n s is the total number of training samples. Training samples are generated w.r.t. the geometrical parameters while frequency is a separate variable swept by the EM simulator during data generation. In our method, frequency is an additional input of the hybrid-based transfer function.
Vector fitting process [24] is performed to obtain a group of poles and residues for each geometrical sample. In the vector fitting process, the given information is EM data d versus frequency for a certain geometrical sample. Expected solutions are poles and residues of the transfer function. Let c k represent a vector containing the poles p (k) and residues r (k) of the transfer function for the kth geometrical sample obtained after vector fitting, defined as We propose a hybrid parameter extraction technique to identify the discontinuous poles/residues, the smoothcontinuous poles/residues, and the non-smooth-continuous poles/residues. After the identification, the smoothcontinuous poles/residues will be maintained into the poleresidue-based transfer function, while the discontinuous poles/residues and the non-smooth-continuous poles/residues together will be re-formulated into rational-based transfer function.
Suppose that we have already obtained the pole and residue data (i.e., p (k) and r (k) ) for all the geometrical samples after vector fitting. Since each pole p (k) i usually has a related residue r (k) i after vector fitting, we suppose that the residue data r (k) are listed in the same way as their corresponding pole data p (k) . The proposed hybrid parameter extraction technique consists of three stages.
The first stage of the proposed hybrid parameter extraction technique is to identify the discontinuous poles/residues and to identify the continuous poles/residues for further 93926 VOLUME 8, 2020 identification in the second stage. Let N be the total order of the hybrid-based transfer function. Let n s be the total number of the geometrical samples. Usually, the discontinuous residues appear together with their corresponding discontinuous poles. Similarly, the continuous residues appear together with their corresponding continuous poles. For simplicity of representation, we use n R i to represent the number of the geometrical samples where the ith pole p i ) is a real pole (or a real residue), i.e., i ∈ I = {1, 2, . . . , N }, where I is the index set of samples of pole data p (k) (or residue data r (k) ). Let η i represent the ratio between n R i and n s for p Here we define a step function f (·) for η i , expressed as i ) is a real pole (or residue) for some geometrical samples and is a complex pole (or residue) for other geometrical samples. In this case, p i ) is either a real pole (or residue) or a complex pole (or residue) for all the geometrical samples. In this case, p i ) is a continuous pole (or residue) and f (η i ) = 0. Let N a 2 be the number of discontinuous poles (or residues), formulated as The index set I a 2 of the samples of discontinuous pole data (or discontinuous residue data) are formulated as Subsequently, the index set I of the samples of continuous pole data (or continuous residue data) can be expressed as I = I − I a 2 . Both index sets I a 2 and I are the subsets of the index set I . So far, we have identified the discontinuous pole/residue data and the continuous pole/residue data through the calculation of N a 2 , I a 2 , and I . In the second stage, we identify the non-smooth continuous poles/residues from the identified continuous poles/residues with the index set I . Here we use σ and where µ p i and µ r i are expressed as and Here we define a shared deviation σ i for both ith continuous poles and residues p i and r i in order to perform the identification, formulated as where i ∈ I . Let σ 0 be the user-defined threshold of the deviation. We define a step function g(·) for σ i , expressed as If σ i > σ 0 , the ith continuous poles and residues p i and r i are considered non-smooth and g(σ i ) = 1. Otherwise, the ith continuous poles and residues p i and r i are considered smooth and g(σ i ) = 0. Let N b 2 represent the number of the non-smooth-continuous poles/residues, formulated as It is noticed that the order N 2 of rational-based transfer function cannot be high, since the transfer function becomes more sensitive to its coefficients if the order is higher. For this reason, N 2 should be less than a certain user-defined order N max 2 (e.g., N max 2 equals eight). Suppose that the number of discontinuous poles (or residues) N a 2 is less than N max 2 . Therefore, N b 2 should satisfy the condition If this condition is not satisfied, then we increase the value of the user-defined threshold of the deviation σ 0 , recalculate N b 2 and recheck the condition with the updated N b 2 . If the condition is satisfied, we obtain the index set I b 2 of the samples of non-smooth-continuous pole data and non-smoothcontinuous residue data, formulated as Subsequently, the index set I 1 of the samples of smoothcontinuous pole data and smooth-continuous residue data can be expressed as I 1 = I − I b 2 . Both index sets I b 2 and I 1 are the subsets of the index set I . So far, we have identified the smooth-continuous pole/residue data and the non-smoothcontinuous pole/residue data through the calculation of N b 2 , I b 2 , and I 1 .
The order N 2 is subsequently calculated as (31) VOLUME 8, 2020 The order N 1 is subsequently calculated as The total index set I 2 of the samples of the discontinuous pole/residue data and the non-smooth-continuous pole/residue data is expressed as I 2 = I a 2 ∪ I b 2 . In the third stage, we gather the discontinuous poles/ residues and the non-smooth-continuous poles/residues with the index set I 2 from the original pole/residue data into the new pole/residue data. These new pole/residue data are reformatted into the coefficient data for the rational-formatbased transfer function. After reformatting, the coefficient data are defined as a Let c k represent a vector containing the hybrid parameters of the hybrid-based transfer function for the kth geometrical sample, defined as where p (k) is a vector of smooth-continuous poles for the kth geometrical sample, defined as N 2 ] T . The stepwise algorithm of the proposed three-stage hybrid parameter extraction technique is illustrated by a flow diagram as shown in Fig. 2.

D. PRELIMINARY TRAINING OF NEURAL NETWORKS
We propose a three-stage training process. In the first stage, we perform a preliminary training process for characterizing the relationships between the poles/residues of the transfer function and the geometrical parameters. During this process, neural networks are trained to learn these relationships. The training data for this phase is (x k , p (k) ) and (x k , r (k) ), k ∈ T r (i.e., geometrical parameters as model inputs and poles/residues as model outputs). The nonlinear relationship between poles and geometrical parameters are usually different from that between residues and geometrical parameters. Subsequently, poles and residues are provided by separate neural networks.
In the second stage, we perform a preliminary training process for characterizing the relationships between the coefficients of the transfer function and the geometrical parameters. During this process, different sets of neural networks are trained to learn these relationships. The training data for this phase is (x k , a (k) ) and (x k , b (k) ), k ∈ T r (i.e., geometrical parameters as model inputs and coefficients as model outputs). The nonlinear relationship between the coefficients in the numerator and geometrical parameters are usually different from that between the coefficients in the denominator and geometrical parameters. Similarly, separate neural networks are used to train the coefficients in the numerator and denominator. During the two stages of preliminary training, a certain amount of training error (e.g., 5%-10%) can be tolerated. With this relaxed error criteria, less hidden neurons can be used in the neural networks. The neural networks thus have lower nonlinearity which makes the hybrid-based neuro-TF more robust. After preliminary training of the neural networks, an overall model refinement process is performed to further refine the preliminary-trained model.

E. REFINEMENT TRAINING OF THE HYBRID-BASED NEURO-TF MODEL
In the third stage of the training process, a model refinement is performed to further refine the overall hybrid-based neuro-TF model. The training data for this phase is (x k , d k ), k ∈ T r (i.e., geometrical parameters as model inputs and EM responses (i.e., S-parameters) as model outputs). The mechanism for the refinement training process of the overall hybrid-based neuro-TF model is shown in Fig. 4. It consists of the hybrid-based transfer function of (2) and the neural networks whose initial values are the optimal solutions from the two-stage preliminary training. This model refinement process consists of both training and testing of the model. The objective of training is to minimize the training error of the overall model. Training is performed by optimizing the weights inside the neural networks to minimize the error function where T r is the index set of training samples of various geometrical parameters, and n s is the total number of training samples. is the index set of frequency samples. w p , w r , w a and w b represent the weights in the neural networks for poles p, residues r, coefficients in numerator a, and coefficients in denominator b, respectively. y represents the outputs of the overall model, which ultimately is a function of geometrical variables x k , frequency s i , and neural network weights w p , w r , w a and w b .
The training process terminates when the training error becomes lower than a user-defined threshold E t . After the training process, an independent set of testing data which are never used in training is used for testing the quality of the trained hybrid-based neuro-TF model. The testing error E Ts is defined as the error between the model response and the testing data. If the testing error is also lower than the threshold error E t , the model refinement process terminates and the hybrid-based neuro-TF model is ready to be used for highlevel design. Otherwise, the overall model training process will be repeated with different numbers of hidden neurons.
A flow diagram illustrating the overall hybrid-based neuro-TF model development process is shown in Fig. 3.

IV. APPLICATION EXAMPLES A. PARAMETRIC MODELING OF A FOUR-POLE WAVEGUIDE FILTER
In this example, the development of a parametric model for the EM behavior of a four-pole waveguide filter [25] using the proposed hybrid-based neuro-TF technique is illustrated. As shown in Fig. 5, the geometrical parameters of the fourpole waveguide filter are In the proposed technique, a hybrid-based neuro-TF model is developed as the parametric model for the four-pole waveguide filter example. As defined in the model structure for the four-pole waveguide filter example in Fig. 6, the model has five input geometrical variables and a frequency variable (as an additional input). The model has two outputs, VOLUME 8, 2020  i.e., y = [RS 11 IS 11 ] T , which are the real and imaginary parts of S 11 , respectively.
The ANSYS HFSS EM simulator is used to perform the fullwave EM simulation for generation of training and testing data for parametric modeling. A parallel computation method is used in data generation. Message passing interface (MPI) is used for parallel processing with distributed memory. Design of experiments (DOE) [26] method is used as the sampling method to sample both training data and testing data.
As can be seen in Table 1, the proposed parametric modeling technique is applied to two different cases, in terms of the parameter range. Case 1 considers a narrower parameter range and Case 2 considers an increased parameter range. In both cases, the total order N of the hybrid-based transfer functions is set to twelve for all the training and test samples. The number of frequency samples for each geometrical sample is 101. In both cases, nine levels of DOE are used to sample the training data, resulting in a total number of 81 samples, while eight levels of DOE are used to sample the testing data, resulting in a total number of 64 samples. The detailed ranges of training data and testing data for the two cases are listed in Table 1. The original frequency range in this example is 10.5 GHz -11.5 GHz. Scaling and shifting of the frequency range are performed. The total order N is split into the order N 1 93930 VOLUME 8, 2020 of the pole-residue-based transfer function and the order N 2 of the rational-based transfer function by using the proposed hybrid parameter extraction technique. Based on the pole/residue data (which are directly obtained from the vector fitting of training data), we identify the discontinuous poles/residues, non-smooth-continuous poles/residues, and the smooth-continuous poles/residues for both cases. The pole-residue data in Case 1 are relatively smooth as the values of the geometrical parameters change. By contrast, the pole-residue data in Case 2 have discontinuity issue in some of the imaginary parts of poles and residues, and have the accompanied non-smoothness issue in other poles and residues, as the values of the geometrical parameters change. The discontinuity issue and the non-smoothness issue in Case 2 can be observed in Fig. 7. As shown in the figure, the original poles/residue data have the discontinuity issue and the non-smoothness issue. After using the proposed extraction technique, the newly obtained rational data have better smoothness than the original poles/residue data. Based on the proposed hybrid parameter extraction technique illustrated in Section III-B, N 1 is set to eight and N 2 is set to four in Case 1. N 1 is set to six and N 2 is set to six in Case 2. The hybridbased neuro-TF model is trained using the NeuroModelerPlus software for both preliminary training and refinement training. The average training error is 0.909% in Case 1 and the average testing error is 1.620% in Case 2.
For comparison purpose, the recent pole-residue-based neuro-TF modeling method and the existing rational-based neuro-TF modeling method [23] are applied to the two cases. Table 2 shows the comparisons of ANN structures, number of hidden neurons, and average training and testing errors between several parametric modeling methods and EM data.
In Case 1, since the geometrical parameters vary within a small range, discontinuity and non-smoothness issues do not exist and the pole-residue data are relatively smooth. Both the pole-residue-based neuro-TF method and the proposed method obtain comparatively small training and testing errors. The rational-based neuro-TF method obtains relatively large training and testing errors, since the order of the transfer function is relatively high.
In Case 2, the geometrical parameters vary within an increased range and the pole/residue data have the discontinuity issue and the accompanied non-smoothness issue. The pole-residue-based neuro-TF method encounters the discontinuity issue and the non-smoothness issue, resulting in large testing error. This is because that the pole/residue data (which are used in the preliminary training) have the discontinuity and non-smoothness issues. These issues are also the cause for the high nonlinearity of pole/residues from sample to sample. To achieve good training result, the number of hidden neurons is increased from 10 to 50. Such increase of the number of hidden neurons, however, leads to the overlearning of the ANN models, making the testing error even worse. The rational-based neuro-TF method obtains relatively smaller testing error than that obtained by the pole-residue-based neuro-TF method with the same number of hidden neurons. This is because that the rational data (which are converted from the pole-residue data for preliminary training) have better smoothness. However, the order of the rational-based transfer function is relatively high. The rational-based transfer function is more sensitive to its coefficients, and the coefficients are more nonlinear than the pole/residues over the increased geometrical parameter range from sample to sample. Therefore, the testing error of the rational-based FIGURE 7. The original poles/residue data with discontinuity issue and the non-smoothness issue versus the newly obtained rational data with improved smoothness (after using the proposed hybrid parameter extraction technique) in Case 2 for the four-pole waveguide filter example. (a) the imaginary part of p 1 versus the training sample from the original poles/residue data, (b) the real part of r 3 versus the training sample from the original poles/residue data, (c) a 2 of the numerator versus the training sample from the newly obtained rational data. neuro-TF model is still high. These are the reasons why the parametric models trained by the two existing methods cannot obtain good training and testing accuracy.
The proposed hybrid-based neuro-TF method can achieve better accuracy in training and testing than the two existing methods for both cases, as shown in the comparison in Table 2. It can be also noticed that the amount of hidden neurons used in the hybrid-based neuro-TF model is small in both cases. This indicates that the good learning of the model is achieved, leading to good training and testing results. The reason behind is that the poles/residues which have the discontinuity and non-smoothness issues in the original pole/residue data are reformatted into the new rational data (of which the smoothness is improved) with less order. The poles/residues which are smooth and continuous in the original pole/residue data are gathered together to format into the new pole/residue data (which are smooth) with less order. These new pole/residue data are used to train the poleresidue-based neuro-TF model and the new rational data are used to train the rational-based neuro-TF model during the preliminary training. Since there is no discontinuity and nonsmoothness issues in the new pole/residue data, the poleresidue-based neuro-TF model can achieve good learning with small amount of hidden neurons. Since the new rational data are smoother and with less order, the rational-based transfer function is less sensitive to its coefficients, leading to good learning of the rational-based neuro-TF model with small amount of hidden neurons as well. The preliminary trained pole-residue-based neuro-TF and rational-based neuro-TF models are then assembled into the hybrid-based neuro-TF model for the refinement training. These are the reasons why the parametric models trained by the proposed method can obtain good training and testing accuracy. Fig. 8 shows the output S 11 of the proposed hybridbased neuro-TF model for three different test geometrical samples in the four-pole waveguide filter example. The S 11 response from the proposed model is compared  It is observed from Fig. 8 that the model using the proposed hybrid-based neuro-TF method can achieve good model accuracy for different geometrical samples even though these test samples are never used in training.

B. PARAMETRIC MODELING OF A THREE-POLE H-PLANE FILTER
In this example, the development of a parametric model for the EM behavior of a three-pole H -plane filter [27] using the proposed hybrid-based neuro-TF technique is illustrated. As shown in Fig. 9, the geometrical parameters of the three-pole H -plane filter are x = [L 1 L 2 W 1 W 2 ] T with a = 19.05 mm, b = 9.525 mm, and t = 2.0 mm. In the proposed technique, a hybrid-based neuro-TF model is developed as the parametric model for the three-pole H -plane filter example. As defined in the model structure for the three-pole H -plane filter example in Fig. 10, the model has four input geometrical variables and a frequency variable (as an additional input). The model has two outputs, i.e., y = [RS 11 IS 11 ] T , which are the real and imaginary parts of S 11 , respectively.
The ANSYS HFSS EM simulator is used to perform the fullwave EM simulation for generation of training and testing data. A parallel computation method is used in data generation. DOE method is used as sampling method.
As can be seen in Table 3, the proposed parametric modeling technique is applied to two different cases, similar to the four-pole waveguide filter example. Case 1 considers a narrower parameter range and Case 2 considers an increased parameter range. In both cases, the total order N of the hybrid-based transfer functions is set to ten for all the training  and test samples. The number of frequency samples for each geometrical sample is 71. In both cases, seven levels of DOE are used to sample the training and testing data, resulting in a total number of 49 training samples and 49 testing samples. The detailed ranges of training data and testing data for the two cases are listed in Table 3.
The original frequency range in this example is 11.6 GHz -12.4 GHz. Scaling and shifting of the frequency range are performed. The total order N is split into the order N 1 of the pole-residue-based transfer function and the order N 2 of the rational-based transfer function by using the proposed hybrid parameter extraction technique. Based on the pole/residue data (which are directly obtained from the vec-tor fitting of training data), we identify the discontinuous poles/residues, non-smooth-continuous poles/residues, and the smooth-continuous poles/residues for both cases. The pole-residue data in Case 1 are relatively smooth as the values of the geometrical parameters change. By contrast, the pole-residue data in Case 2 have discontinuity issue in some of the imaginary parts of poles and residues, and have the accompanied non-smoothness issue in other poles and residues, as the values of the geometrical parameters change. The discontinuity issue and the non-smoothness issue in Case 2 can be observed in Fig. 11. As shown in the figure, the original poles/residue data have the discontinuity issue and the non-smoothness issue. After using the proposed extraction technique, the newly obtained rational data have better smoothness than the original poles/residue data. Based on the proposed hybrid parameter extraction technique illustrated in Section III-B, N 1 is set to six and N 2 is set to four in both cases. Similarly, the hybrid-based neuro-TF model is trained using the NeuroModelerPlus software for both preliminary training and refinement training. The average training error is 0.356% in Case 1 and the average testing error is 1.729% in Case 2.
For comparison purpose, the recent pole-residue-based neuro-TF modeling method and the existing rational-based neuro-TF modeling method are also applied to the two cases of this example. Table 4 shows the comparisons of ANN structures, number of hidden neurons, and average training and testing errors between several parametric modeling methods and EM data.
In Case 1, since the geometrical parameters vary within a small range, discontinuity and non-smoothness issues do not exist and the pole-residue data are relatively smooth. The order of the transfer function is moderate (not too high). . The original poles/residue data with discontinuity issue and the non-smoothness issue versus the newly obtained rational data with improved smoothness (after using the proposed hybrid parameter extraction technique) in Case 2 for the three-pole H-plane filter example. (a) the imaginary part of p 1 versus the training sample from the original poles/residue data, (b) the real part of r 7 versus the training sample from the original poles/residue data, (c) a 2 of the numerator versus the training sample from the newly obtained rational data.
For these reasons, all the methods in this case achieve comparatively small training and testing errors.
In Case 2, the geometrical parameters vary within an increased range and the pole/residue data have the discontinuity and non-smoothness issues. The pole-residue-based neuro-TF method, again, encounters these issues, resulting in large testing error. To achieve good training result, the number of hidden neurons is increased from 5 to 40. Such increase of the number of hidden neurons, however, leads to the overlearning of the ANN models, making the testing error even worse.
The rational-based neuro-TF method obtains relatively smaller testing error than that obtained by the poleresidue-based neuro-TF method. However, the order of the rational-based transfer function is not low enough and the geometrical variations are larger than those in Case 1. The rational-based transfer function is still more sensitive to its coefficients, and the coefficients are more nonlinear than the pole/residues over the increased geometrical parameter range from sample to sample. Therefore, the testing error of the rational-based neuro-TF model is still not satisfactory.
The proposed hybrid-based neuro-TF method can achieve better accuracy in training and testing than the two existing methods for both cases, as shown in the comparison in Table 2. It can be also noticed that the amount of hidden neurons used in the hybrid-based neuro-TF model is small in both cases. This suggests that the good learning of the model is achieved, leading to good training and testing results. In the proposed technique, there is no discontinuity and non-smoothness issues in the new pole/residue data and the new rational data are smoother and with less order. The poles/residues which are non-smooth and discontinuous in the original pole/residue data are gathered together to reformat into the new rational data (which are much smoother) with less order. Since the new rational data are smoother and with less order, the rational-based transfer function is less sensitive to its coefficients. Therefore, both the pole-residue-based neuro-TF model and the rationalbased neuro-TF model can achieve good learning with small amount of hidden neurons, leading to good training and testing accuracy of the hybrid-based neuro-TF model.  It is observed from Fig. 12 that the model using the proposed hybrid-based neuro-TF method can achieve good model accuracy for different geometrical samples even though these test samples are never used in training. As additional results, we also include the comparison of real and imaginary parts of S 11 using different modeling methods in Fig. 13. It can also be seen from this figure that the model using the proposed hybrid-based neuro-TF method can achieve good model accuracy.
So far the focus of our work is on addressing the discontinuity issue and the accompanied non-smoothness issue in the poles/residues with the same order. Discontinuity issue coming from order changing is another situation to be considered. Order changing is needed for parametric modeling especially when different responses cannot be effectively represented with a same order from vector fitting, as the geometrical variables change. It would be an interesting future direction to consider the case of the discontinuity and on-smoothness issues coming from order changing.

V. CONCLUSION
This paper has presented a novel hybrid-based neuro-TF parametric modeling technique which systematically combines both pole-residue and rational formats of the transfer functions, addressing the discontinuity and non-smoothness issues. The proposed technique identifies the discontinuous poles/residues, the smooth-continuous poles/residues, and non-smooth-continuous poles/residues. The discontinuous and non-smooth-continuous poles/residues are gathered together to reformat into the coefficients of the rationalbased transfer function to effectively address the discontinuity and non-smoothness issues. The smooth-continuous poles/residues are gathered together to be remained in the pole-residue format of the pole-residue-based transfer function to maintain the capability of handling high-order problem. In this way, the discontinuity and non-smoothness issues are solved by the proposed hybrid-based neuro-TF modeling technique and high-order modeling capability is maintained by the overall model. The developed model can provide accurate and fast prediction of the EM behavior of microwave components. As illustrated in the examples and compared with the existing modeling methods, the proposed technique can obtain better accuracy in challenging applications of large geometrical variations and high order, addressing the discontinuity and non-smoothness issues.