A Novel Method for Estimating Residual Model Parameters to Evaluate Uncertainty in Scattering Parameter Measurements

In this article, we present a novel method for estimating the parameters of residual models, which are widely used to evaluate the measurement uncertainty of the scattering parameters measured by a vector network analyzer (VNA). Conventionally, the parameters of the residual model are obtained using the ripple method with an air line, where the insulator that distinguishes the inner and outer lines of the coaxial line is air. However, due to inaccuracies caused by losses from air lines, the usable range of frequencies is limited. In addition, methods using the invariance property of the cross ratio of complex numbers result in overestimation because the maximum error boundary is calculated. Above all, since both methods estimate only the magnitude uncertainty, the phase uncertainty estimation is not rigorous, and the correlation between the magnitude and the phase cannot be estimated. In contrast, the proposed method calculates the parameters of the residual model directly from the calibration standards uncertainty, including the phase as well as magnitude, achieving the same result as the VNA error model. The proposed parameter estimation for residual models can also easily propagate to the uncertainty of other measurements since it can obtain the correlation between magnitude and phase as well as the cross frequency. In this article, we compared the parameters of the residual model using the proposed method, the conventional ripple method, and the invariance property of the cross ratio of complex numbers. For validation, the uncertainty obtained using the residual model with the proposed parameter estimation was compared to the results from the VNA error model. The two uncertainties showed excellent agreement for highly reflective and matched devices. Finally, we discuss how to effectively manage the calibration and measurement capabilities of scattering parameters using the proposed uncertainty evaluation method.


I. INTRODUCTION
A VECTOR network analyzer (VNA) is one of the most used apparatuses for microwave measurement. In particular, VNAs are widely used to measure scattering parameters (S-parameters) since they can simultaneously measure the magnitude and phase of the incident and reflected waves. Although a VNA is designed to have a frequency response near 1 and its inner circuits to have low reflection, there are naturally systematic errors in the VNA measurements when operating over a wide frequency range. In addition, the cable used to connect a device under test (DUT) to the VNA introduces additional errors. As a result, various calibration methods have been developed and applied to calibrate these systematic errors. Also, uncertainty evaluation methods have been proposed for each of these calibration methods.
The uncertainty evaluation methods can be classified into VNA error models and residual models. The VNA error model is essentially a calibration model to correct the raw measurement data. Thus, the VNA error terms (usually directivity e 00 , source match e 11 , and reflection tracking e 10 e 01 for one-port calibration) should be determined in advance. The S-parameters of a DUT are then calibrated using the determined VNA error terms. As a result, the uncertainty of the VNA error term is calculated first, and the uncertainty of the DUT is calculated using this. Accordingly, measurers require a high level of knowledge, and in some cases, they need to implement a calibration algorithm to calculate the uncertainty.
Meanwhile, the residual models analyze measurement uncertainty using a calibrated result. In other words, the residual model is easy to calculate because it does not include a calibration process. Sometimes, the manufacturers of the VNA and calibration standards offer residual errors to help users estimate measurement uncertainty.
The ripple method is a typical approach for estimating parameters in a residual model. It estimates the parameters from the ripple observed after connecting the air line between the test port of the VNA and high-or low-reflection materials, such as a short or a load calibration standard. An air line is a low loss coaxial line where the insulator between the inner and outer conductors is air. However, as the operating frequency increases, the conductor loss in the air line increases and unwanted reflections from the connector joints occur. For this reason, it is recommended to use the ripple method at frequencies below 26.5 GHz [1].
Another approach utilizes the invariance property of the cross ratio of the complex number and estimates the maximum error boundary [2], [3], [4]. However, both methods can only calculate the magnitude of the uncertainty of the DUT and estimate phase uncertainty from the magnitude uncertainty using simple geometry. Thus, the correlation between the magnitude and phase is not accounted for so that the measurement uncertainty is underestimated or overestimated. In [5], the uncertainty was calculated for the short-open-load-thru (SOLT) and short-open-load-reciprocal (SOLR) calibration methods using a residual model. However, the detailed method used to obtain the residual parameters is not described. In [6], the residual error was calculated using the air line and offset short in the time domain. The measured reflection coefficient should be separated into individual residual errors with a filter.
Recently, we calculated the uncertainty of the electronic calibration unit (ECU) using a residual model in [7]. However, we could not deal with the method for calculating the residual error in detail. In this article, we will explain a novel method to obtain the parameters of the residual model from the uncertainty of the calibration standards. The proposed method can easily calculate the parameters using a Monte Carlo simulation or sensitivity analysis since it applies a linearly transformed one-port calibration algorithm. In addition, when applying the proposed approach, we can obtain the correlation of complex numbers by dividing the S-parameters of the DUT measured after VNA calibration into the real/imaginary parts or magnitude/phase. This article describes a novel method for converting the uncertainty of SOL calibration standards into residual errors. In addition to the SOL calibration standards, various standards can be additionally used, such as sliding load [8]. Although this article does not go into detail about this, it is possible to convert the uncertainty of these calibration standards into residual errors.
This article is organized as follows. Section II explains the VNA error model and the residual model. In Section III, we derive equations for the proposed method based on the sensitivity analysis. In Section IV, we show some examples of measurement uncertainty using the residual model after obtaining the parameters. In Section V, we compare the proposed and the previous methods for obtaining the parameters of the residual model. We also show another application of the residual model for managing calibration and measurement capability (CMC) in Section VI, and the conclusion is provided in Section VII. Fig. 1(a) shows the VNA error model for one-port calibration. Here, e 00 , e 11 , and e 10 e 01 are the error terms of the VNA and mean directivity, source match, and reflection tracking, respectively [3]. Thus, the measured scattering parameter M at the input port of the VNA is given as follows:

A. VNA Error Model
where a 0 and b 0 are the incident and reflected waves at the input port, respectively, and e = e 00 e 11 − e 10 e 01 . Therefore, to calibrate the scattering parameter of the DUT, the error terms of the VNA must be known. To determine the unknown e 00 , e 11 , and e 10 e 01 , we require at least three or more calibration standards with impedance values known in advance. In this article, our explanation assumes the short, open, and load (SOL) calibration standards that are widely used in calibration processes. We can represent (1) as three unknown simultaneous equations as follows: where S , O , and L and S Meas , O Meas , and L Meas are the definition and measurement values of the reference calibration standards of short, open, and load, respectively. This equation is a linear least-squares (LSQ) problem, and the general solution is given as follows: where H is a conjugate transpose. Then, the reflection coefficient of the DUT can be determined using The previous procedure is repeated to estimate the measurement uncertainty of the DUT. For example, when performing the Monte Carlo simulation, a number of N short, open, and load calibration standards are artificially created according to their uncertainty. In order to calculate the VNA error term, the generated values of each standard are substituted into (3). Then, the reflection coefficient of the DUT is determined using a number of N sets of the VNA error terms. Finally, the uncertainty is estimated from the N set of . As described above, for the VNA error model, after determining the VNA error terms using the calibration standard, the value of the DUT should be calculated. In the same way, when calculating the uncertainty, it is necessary to propagate the uncertainty of the calibration standards as the uncertainty of the VNA error term and then subsequently propagate it to the uncertainty of the DUT. Fig. 1(b) represents the residual model. This model assumes the calibrated VNA, resulting in the error terms e 00 and e 11 that are 0 and e 10 e 01 that is 1, which is significantly different from the VNA error model. As a result, the reflection coefficient C of the DUT at the input terminal is equal to the true reflection of the DUT. The VNA, even if it has been calibrated, has residual errors δ, μ, and τ due to the uncertainty of the calibration standards and random errors of the VNA. These residual errors produce the uncertainty in the DUT. The residual model was originally designed to account for the uncertainty of the calibrated VNA. In other words, the residual model enables us to easily obtain the uncertainty of the DUT since it does not include the calibration process. However, despite their usefulness, it is difficult to precisely obtain the residual errors.

B. Residual Model
The proposed method first modifies (2) to establish the calibrated VNA ⎡ That is, S Meas , O Meas , and L Meas are replaced with S , O , and L so that M at the input terminal and of the DUT are the same. As a result, the VNA error terms e 00 and e 11 are 0 and e 10 e 01 is 1 in (5). If we generate the values of the N set of calibration standards depending on their uncertainty, we can use (6) to get an N set of e 00 , e 11 , and e 10 e 01 ⎡ ⎢ ⎣ Here, S i , O i , and L i refer to the i th calibration standards among N calibration standards, that is, the measurement values of the calibration standards are assigned according to their definitions so that they become the calibrated VNA. The definitions change according to their uncertainty and propagate their uncertainty to the calibrated VNA error term. Thus, each mean of e 00,i , e 11,i , and e 10 e 01,i is 0, 0, and 1, and their deviations can be assigned to the residual errors δ, μ, and τ , respectively [7].
The proposed method can be easily realized using a Monte Carlo simulation. In the simulation, we can obtain the correlations real/imaginary parts or magnitude/phase and cross frequencies by setting the error term as desired forms when calculating the deviations in the error terms. For example, if we change the form of the obtained error terms to be [|e 00,i ( f 1 )|, |e 11,i ( f 1 )|, |e 10 e 01,i ( f 1 )|, |e 00,i ( f 2 )|, . . . , |e 10 e 01,i ( f N )|], then the covariance for the magnitude, including the cross frequencies, can be obtained.

III. CALCULATING RESIDUAL ERRORS BASED ON A SENSITIVITY ANALYSIS
Generally, guide to the expression of uncertainty in measurement (GUM) recommends more than 200 000 simulations for the Monte Carlo simulation [9]. An uncertainty propagation based on a sensitivity analysis takes significantly less time than a Monte Carlo simulation. This section describes how to calculate the residual error using a sensitivity analysis.
The uncertainty propagation can be expressed using the first-order Taylor series according to the GUM This equation can be simplified when N = 2, and it can be represented as a product of the matrix This can be generalized as the product of the Jacobian matrix J and the covariance matrix as follows [10], [11]: where is the transpose matrix. In this section, the sensitivity analysis is based on the above formula. We should calculate the Jacobian matrix J cal.VNA to propagate the covariance of the calibration standards std to the residual covariance δμτ . J cal.VNA is a matrix composed of the deviation of the calibrated VNA, that is, the amount of change in δ, μ, and τ when the calibration standards are changed. To obtain the sensitivity coefficient, we first need to find the relation between the calibration standards and the residual.
In (3), we replace S Meas , O Meas , and L Meas with S , O , and L to make the calibrated VNA. Therefore, e 00 , e 11 , and e 10 e 01 become 0, 0, and 1, respectively. Now, the sensitivity coefficient can be obtained by the partial differentiation of (3). When calculating partial derivatives, the substituted S , O , and L should remain constant. For example, when calculating ∂e 00 /∂ S , the substituted S in the 3rd, 7th, 11th, and 13th columns of the right of (3) is a constant, not a variable (see the Monte Carlo summation in Section III).
Also, we will calculate the covariance of the complex value by dividing it into real and imaginary parts. With this distinction, the correlation between the real and imaginary parts can also be computed and easily converted to another quantity (e.g., magnitude and phase). Thus, the Jacobian J cal.VNA is given as in (11), shown at the bottom of the page, where the arguments of (·) and (·) mean the real and imaginary parts of the parameters. To match with (11), the covariance of the calibration standards std should have a 6 × 6 array as in (12), shown at the bottom of the page, σ represents the standard deviation. Thus, the diagonal matrix is in the order of the square of the standard deviations of the real and imaginary parts of the three calibration standards, i.e., short, open, and load. Now, we can obtain the covariance of the residual error δμτ as follows: In (13), the parameters δ, μ, and τ of the residual model do not represent specific values but rather indicate the variance in the calibrated VNA due to the uncertainty of the calibration standards. Unlike Monte Carlo simulations, a sensitivity analysis may have a bias in the results depending on the distribution of random numbers and the presence of high-order terms since it is based on the first-order Taylor series expansion. Fortunately, we confirmed that there was no such bias when the residual error was calculated using the sensitivity analysis. Fig. 2 shows the standard deviation in the residual error calculated using the Monte Carlo simulation and the sensitivity analysis. It can be seen that the two results agree well with each other. This means that the sensitivity analysis described in this section to calculate the residual error can replace the Monte Carlo simulation.

IV. VALIDATION OF THE PROPOSED RESIDUAL MODEL
In this section, we measure high reflective and low reflective DUTs and compare the uncertainties of the DUTs using the residual model, applying the proposed method and the VNA error model.
As described in Section III, the uncertainty can be propagated through the product of the covariance and the Jacobian matrix. Thus, the covariance of the DUT based on the VNA error model is given as follows: The Jacobian matrix J std propagates the covariance of the calibration standards std to the covariance of the VNA error term, and J Err propagates it to the covariance of the  DUT DUT,ErrModel . Each Jacobian matrix has 6 × 6 and 2 × 6 elements as follows: Finally, the covariance of the DUT DUT,ErrModel has a 2 × 2 matrix. As a result, we can obtain the measurement uncertainty with a 95% confidence interval by taking the square root of the diagonal matrix from the calculated covariance and multiplying it by a proper coverage factor (e.g., k = 2 for normal distribution). To consider the correlation between cross frequencies, (14)-(16) can be extended to include other frequency components. Similarly, we can calculate the covariance DUT,Res of the residual model as follows: Here, J residual is formed by substituting δ, μ, and τ for e 00 , e 11 , and e 10 e 01 in (15). The covariance δμτ of the residual model parameters can be obtained through the Monte Carlo simulation, as described in Section II, and the sensitivity analysis is described in detail in Section III. Fig. 3 shows the DUT uncertainty (k = 2, 95% level of confidence) of the VNA error model and the residual model calculated using (14) and (17), respectively. The coverage factor k depends on the distribution. We assumed that the uncertainty of the calibration standards has a normal distribution and confirmed that the uncertainty of the DUT also has a normal distribution using a Monte Carlo simulation. We normalize the phase to 0 for easy comparison. This shows that the uncertainty of the DUT calculated using the proposed residual model exactly matches the VNA error model regardless of the DUTs' reflection. Thus, by using the proposed residual error calculation method, we can simultaneously maintain the residual model's usefulness and accuracy.

V. COMPARISON OF METHODS FOR CALCULATING RESIDUAL ERRORS
The classic approach for obtaining the residual errors is the ripple method. This method observes the ripple after one port of the air line is connected to the calibrated VNA and the other port is connected to a low reflective device such as a load calibration standard. The ripple can be observed when the air line has low loss, as shown in Fig. 4. Note that the frequency interval at which the ripple repeats varies depending on the length of the air line. As shown in the inset of Fig. 4, each maximum in the ripple is connected with a straight line (dashed line), and then, a line is drawn vertically from the straight line to the minima (blue line). This line is divided by 2, and its value is assigned to the residual directivity error δ. Again, a highly reflective device such as the short calibration standard is attached to the air line, and the residual source match μ is obtained using the same approach from the measured ripple. The ripple method, however, cannot obtain the residual reflection tracking τ and is recommended for use up to 26.5 GHz, due to losses in the air line [1]. Another issue is that the magnitude of the ripple can change depending on the high (or low) reflection devices used.
The cross ratio method calculates the maximum error boundary using the invariance property of the cross ratio of complex numbers [2], [3]. The calculation of the maximum boundary is given as follows: where i is the magnitude uncertainty of the calibration standard of i . Thus, these equations cannot be used for more than three calibration standards (or status), such as ECUs. Also, like the ripple method, it is unable to obtain the correlation between the magnitude and the phase. Fig. 5 presents the consequential residual errors using each method. The dashed line with circle, dashed-dotted line, and solid line are the results using the ripple method, the cross ratio method using (18)-(21), and the proposed method, respectively. The ripple method exhibits overestimated or underestimated results compared to the other two approaches. Also, the ripple method can only calculate at the frequency where the ripple is produced. The cross ratio method can achieve dense frequency data but overestimates results since it calculates the maximum error bound. On the other hand, the proposed method can calculate an appropriate residual error in all frequency ranges. Another advantage of the proposed parameter estimation of the residual model is that it becomes easy to manage the CMCs. The CMC is an indicator of the calibration capability of each laboratory. For the S-parameter, the CMC means the smallest uncertainty that can be achieved depending on the frequency and the scattering coefficient of the DUT.
It is difficult to calculate the CMC with the VNA error model because the VNA error term changes depending on the measurement setup. However, since the residual model assumes the calibrated VNA, the uncertainty can be easily calculated by just inputting the DUT, whose uncertainty will be determined in the residual model. For example, Fig. 6 shows the calculated DUT uncertainty when the DUT's scattering coefficient || is 0.5. Here, we plot the uncertainty simultaneously by calculating the DUT phase in 30 • intervals from 0 • to 360 • since the uncertainty changes depending on the phase of the DUT. Therefore, when the DUT has a reflection || of 0.5, the CMC is the red line with the smallest value. As a result, CMC management is easier using the residual model with the proposed parameter estimation than with the VNA error model. Here, to simplify the discussion, only the uncertainty of the calibration standards is included in the residual model. However, as described in [1] and [12], a more rigorous uncertainty evaluation can be achieved by including the VNA random errors (noise floor, trace noise, linearity, and drift), cable movement, and connector repeatability.

VII. CONCLUSION
In this article, we proposed a new method to calculate the parameters of the residual model directly from the uncertainty of the calibration standards. The proposed parameter estimation method produces the same uncertainty results as the VNA error model. As a result, it is easy to calculate the uncertainty while maintaining the same accuracy as the VNA error model. For example, it is possible to inherit the advantages of the residual model, which can use the VNA measurement result as it is. Note that for the VNA measurement results to have measurement traceability, data from the so-called "data-based model" calibration standards should be directly loaded into the VNA. Since the approach of this article can be easily implemented, it is expected that other researchers will be able to utilize it for VNA measurement uncertainty calculations in various laboratories.