Uncertainty Analysis of Scattering Parameters Calibrated by an Electronic Calibration Unit Based on a Residual Model

We propose a new residual model to analyze the uncertainty of scattering parameters (S-parameters) calibrated by an electronic calibration unit (ECU). Residual errors are usually estimated from the observed ripple after connecting a load or a short at the end of an airline. Therefore, this ripple method can only be used in a frequency range where the airline loss was not large. We, however, obtained the residual error from the uncertainty of the calibration kit using a simple numerical approach. As a result, we can determine the correlations between real/imaginary and magnitude/phase uncertainties. The proposed residual model showed the same results as a VNA error model. We also added a new error term to account for the effect of temperature-dependent drift of the ECU. In addition, we analytically derived the sensitivity coefficients for a 2-port DUT based on the proposed residual model. The proposed residual model will be helpful for the uncertainty analysis of S-parameters calibrated using the ECU.


I. INTRODUCTION
An electronic calibration unit (ECU) 1 is now being widely used for vector network analyzer (VNA) calibration instead of a mechanical calibration kit. The ECU is usually capable of electrically changing 4 to 7 impedance states per ports, and each state is designed to increase the accuracy of the VNA calibration by being distributed as wide as possible on the Smith chart. In this way, the ECU greatly reduces the number of required connections compared to the mechanical calibration kit. For example, on a two port Short-Open-Load-Thru (SOLT) calibration, the mechanical calibration kit requires connection 7 times (3 reference calibration devices per each port, and a 'thru' connection) while the ECU can be calibrated with only one connection. This reduced number of connections minimizes cable movement and improves The associate editor coordinating the review of this manuscript and approving it for publication was Fulvio Schettino . 1 The ECU has various names, such as E-Cal or Automatic calibration unit, depending on the manufacturer. connection repeatability, resulting in reduced measurement uncertainty. However, there have been concerns about longterm stability, since electronics are used in the ECU. Williams et al.. has shown that the ECU is stable enough, comparable to mechanical calibration kits [1]. Thus, it is expected that the ECU will be more widely used.
Recently, some studies have been conducted on the calibration of the ECU with measurement traceability. In [2], the uncertainty of each impedance state of the ECU was propagated to the uncertainty of the VNA systematic error terms, based on the weighted least square fitting (LSQ) algorithm. Then the residual error was calculated using the principle of cross-ratio invariant [3]. The residual error, however, does not consider the correlation between real and imaginary parts, or the magnitude and phase of the complex number, leading to over (or under) estimation of the uncertainty [4]. In [5], the impedance of each state on the ECU was characterized using the Microwave Uncertainty Framework (MUF) developed by the National Institute of Standards and Technology (NIST). Then, this characterized value of the ECU was used to measure the impedance of the device under test (DUT) and estimate the measurement uncertainty. This approach requires post-processing with MUF, since the VNA is just used for raw data collection. In [6], the residual model was applied to calculate the measurement uncertainty of the ECU. This approach obtained the residual uncertainty from the difference between the VNA systematic error terms; one comes from the VNA calibration with the reference calibration kit; the other is from the calibration using the ECU with the data, which is stored in internal memory by the manufacturer. As a result, the uncertainty increases when the data stored in the ECU differs from the reference calibration kit, and there is no way to reduce the uncertainty. In addition, it did not include VNA noise, linearity, cable movement, etc., [7].
In this study, we propose a new residual error model to precisely estimate the measurement uncertainty of the S-parameters calibrated by ECU. This model includes the VNA random error, cable movement, connector repeatability, and the drift of ECU depending on temperature. The advantage of the proposed residual model is that it does not require a calibration process when calculating the uncertainty, just like the conventional residual model. This makes the measuring procedure simple. Additionally, we will show a simple way to numerically obtain the residual model considering the correlation between complex numbers. Thus the proposed residual model can easily calculate the measurement uncertainty while faithfully following the VNA calibration guide on EURAMET [8].
This paper is organized as follows. Section II explains the features of the proposed residual model. Section III describes how to construct a residual model. First, the residual error is calculated from the uncertainty of the ECU. A method for evaluating drift according to the temperature of the ECU is introduced. Section IV provides an example of calculating the S-parameter uncertainty of a 2-port DUT based on the residual model. Finally, this paper is concluded in Section V.

II. FEATURES OF THE PROPOSED UNCERTAINTY MODEL A. ACCURATE RESULTS AND SIMPLE CALCULATION
With the proposed residual model it is possible to simplify the signal flow graph, since it has an expected value of 0 (or 1) of the VNA error terms such as directivity (e 00 ), source match (e 11 ), and reflection tracking (e 10 e 01 ). The VNA error model, however, makes it difficult to simplify the signal flow graph due to the non-zero (or non-ones) error terms. Fig. 1(a) and Fig. 1(b) are the VNA error model and the proposed residual model, respectively. Here, N H , N L , and L represent the trace noise, noise floor, and linearity of the VNA, respectively. Also, D ij represents the drift of the VNA error term. CA T and CA R represent the transmission and reflection stability of the cable, and CO R means the repeatability of the connector [8]. The residual model has residual error δ, µ, τ and temperature drift T ij instead of error terms e 00 , e 11 , and e 10 e 01 . For example, when analyzing cable stability, if the cable stability does not correlate with other errors, it can be simplified as shown in Fig. 1(c) and Fig. 1(d), respectively. The figures show the difficulty the VNA model has with simplification due to the error term. The proposed residual model, however, can simplify until only cable stability CA T and CA R remain. This makes the uncertainty calculation very convenient.
To propagate each uncertainty to the DUT, it is necessary to calculate the sensitivity coefficient. The simplified model can lead to the analytical calculation of the sensitivity coefficient. We derive the sensitivity coefficients of the simplified model in Appendix B.
The proposed residual model shows the same results as the VNA error model even though its calculation is simplified. The ripple method, which has been traditionally used to obtain the residual error, is recommended for applications up to 26. 5 GHz due to losses in the coaxial line. Moreover, it cannot obtain the reflection tracking τ [8]. The proposed VOLUME 10, 2022 method has no limitations as it propagates directly from the uncertainty of the calibration kit to the residual error. Fig. 2 shows a comparison of the uncertainty calculated using the VNA model, with the uncertainty using the proposed residual model. In this figure, the VNA error (e 00 , e 11 , e 10 e 01 ) and the residual error (δ, µ, τ ) are only included for easy comparison. The two results agree well with each other. However, [3] uses the principle of cross ratio to calculate the upper boundary and approximate the residual error. Then it calculates the uncertainty of the DUT from the residual error, as shown in Fig. 2 with the dash-dotted line. The result is more overestimated than the VNA error model or the proposed residual model, since the correlation between the real/imaginary parts or magnitude/phase is not considered.

B. ENABLES RIGOROUS UNCERTAINTY ANALYSIS
We made a residual model accounting for the drift in the ECU due to temperature variation. In [9], Judaschke and Kuhlmann showed that a temperature change of about 2 • C resulted in changes in the ECU's impedance of 1% at 24 GHz. Thus, we added the temperature drift term T ij to the proposed uncertainty analysis model shown in Fig. 1(b). Our results show that the phase uncertainty of the S-parameter can be increased by about 1.5 times. Section III.B describes how to measure drift and convert it to T ij in the residual model. A rigorous uncertainty calculation is possible, including random error terms such as cable movement, connector repeatability, VNA noise, VNA linearity, and VNA drift. For the characterization of these terms, [8] can be referred to. According to our experiments, as the frequency increases, the uncertainty caused by these random errors becomes increasingly dominant and cannot be neglected.
In [5], each state and uncertainty of the ECU were precisely characterized using the airlines and the NIST MUF. However, most secondary calibration labs generally employ a SOLT calibration kit. Thus we describe a method for characterizing the ECU using a calibrated SOLT reference calibration kit in Appendixes C and D.

C. SIMPLIFIES THE CALIBRATION PROCESS
Using the proposed residual error model we do not need an external computer or software to control the ECU state and obtain raw measurements when measuring the DUT. In general, the uncertainty of the DUT is obtained by propagating the uncertainty of the calibration kit to the uncertainty of the VNA error terms, as shown in Fig. 1(a), and then propagate it again to the uncertainty of the DUT. Thus, raw measurements of each ECU impedance state are needed to determine the VNA error terms. The residual method, however, uses the values displayed in the calibrated VNA as they are. That is, the VNA measurement calibrated by the ECU is propagated into in the residual model shown in Fig. 1(b), and calculates the uncertainty.

III. BUILDING THE RESIDUAL MODEL
In this section, we will obtain the residual uncertainty δ, µ, τ from the uncertainty of the ECU's impedance states. The characterization method for each state of the ECU is depicted in Appendix C. In general, ECUs have more than four impedance states. If there are p impedance states, it can be expressed in a linear form as follows.
Here, e = e 00 e 11 -e 10 e 01 , and e 00 , e 11 , and e 10 e 01 are the VNA error terms shown in Fig. 1(a). Also, i (i = 1, 2, . . . , p) is the definition of each ECU state, and Mi (i = 1, 2, . . . , p) is the raw measurement value of the ECU states. When (1) are expressed as AE = M, the general solution of the weighted LSQ for E is as follows.
where H is a complex transpose, W is the weighting factor, σ ij is the covariance for the impedance states of i and j of the ECU. Now, we modify (2) to transform each state of the ECU into the residual model. The residual model assumes an ideal VNA (e 00 = 0, e 11 = 0, and e 10 e 01 = 1). The ideal VNA can be easily made by replacing M to M * = [ 1 , 2 , . . . , p ] . Here, denotes the transpose matrix. We get the relation between ECU states and residual error by replacing M with M * in (2). From this relation, the uncertainty of the ECU can be linearly propagated to the residual uncertainty through a Jacobean matrix and covariance matrix [10]- [13].
where ECU is the covariance of each ECU state evaluated in Appendix D, and δµτ is the covariance of the residual error. All covariances in this paper consist of real and imaginary parts as shown below.
Thus, J ECU −δµτ is a Jacobian matrix obtained through the numerical differentiation of (2) after replacing M with M * as follows.
(e 10 e 01 −1) · · · (e 10 e 01 −1) where (·) and (·) refer to the real and imaginary number of the argument, respectively. Now, we can determine the correlation between the real/image parts of the complex number as shown in (5). Note that δ, µ, and τ are the same as the variances of e 00 , e 11 , and e 10 e 01 -1 in the ideal VNA, respectively. B. DRIFT T ij Fig. 3 represents the measurement setup to account for the drift in the ECU states depending on temperature. Generally, a heater is installed inside the ECU to minimize the effects of external temperature changes, but the ECU state changes slightly with temperature [9], [14]. To accurately count this effect, we thermally isolated the ECU and the outside by employing a temperature chamber. The used chamber was the same as in [15] except that the innermost shield was removed. Although one shield was removed, two proportional-integral-derivative (PID) thermostats were attached on each shield and the outside was shielded with foam, so the temperature was maintained within several mK. In addition, we applied an adiabatic line to prevent thermal flow between the VNA and the ECU [16], [17]. The operating temperature range of the ECU used was (23 ± 3) • C. In this way, the drift in the ECU states was measured as the internal temperature of the chamber was changed within this range. After monitoring the temperature inside the chamber every 5 minutes, it was determined that the internal temperature was stabilized when the temperature change was less than 7 mK. The impedance changes with temperature changes were measured as follows. First, we stabilized the internal temperature of the chamber at 23 • C and calibrated the VNA using the characterized ECU. After setting the temperature to 20 • C, the impedance state of the ECU was measured every five minutes for three hours. Again, we stabilized the internal temperature to 23 • C and calibrated the VNA. This time, we measured the impedance state of the ECU again after changing the chamber temperature setting to 26 • C. The dotted line in Fig. 4 is the impedance of the third state of the ECU A port at 50 GHz measured in this way. We repeated the measurements four times. There is a slight difference in the measurements, but the impedance change due to temperature can be observed. The magnitude differs by about 0.003, and the phase differs by about 1.5 • over the (23 ± 3) • C temperature range. In other words, it is possible to evaluate the uncertainty by considering the effect of drift due to the temperature change in the ECU. The heat flow from the VNA heated by the ECU to the DUT may change the DUT's scattering coefficient. This variation is much smaller than the variation with ECU temperature described in Fig. 4 [9]. Thus, we did not consider this effect in this study. VOLUME 10, 2022 We will now propagate the measured drift of ECU states to T ij in the proposed residual model shown in Fig.1(b). What is important here is that the drifts of each ECU state are strongly correlated with each other. Thus, the uncertainty may be underestimated (or overestimated) when T ij is modeled without this correlation. It is not easy to directly estimate the correlation between the magnitude and phase of each ECU state from Fig. 4. To solve this problem, we first averaged the results of the previous four measurements and then performed a polynomial fit. In the fitting process, we used a 5 th order polynomial. A total of p × 2 × nFr fitting coefficients were obtained per each port. Here, p is the number of ECU states, and nFr is the number of observed frequencies. The solid line in Fig. 4 results from the polynomial fit for the third state of the ECU at 50 GHz. We performed Monte-Carlo simulations to account for all correlations. First, assuming that the temperature change was uniform, we generated 10,000 random numbers between 20 and 26 • C. By inputting the generated random number into the polynomial fit performed earlier, we obtained the impedance state of the ECU depending on temperature. After that, we calculated the error term using (2). At this time, 1 ∼ P used each impedance state of the ECU at 23 • C, and M1 ∼ MP allocates the ECU's impedance states generated from the polynomial fit. The expected values of e 00 , e 11 , and e are 0, 0, and -1. However, e 00 , e 11 , and e 10 e 01 vary due to the changed ECU's impedance state by temperature fluctuation. This variation becomes the temperature drift term T ij of the ECU. Thus the covariance of T ij is obtained from the covariance of e 00 , e 11 , and e 10 e 01 which are calibrated in the Monte-Carlo simulation. The calculated covariances include all variances between (T 00 ), (T 00 ), (T 11 ), (T 11 ), (T 01 ), and (T 01 ). Thus, it can be linearly propagated into the magnitude/phase covariance as shown in Fig. 5 [10]. This figure shows the standard deviations of the magnitude and phase of T ij as a function of frequency, respectively. Note that the phase deviations of T 00 and T 11 are considerable because the magnitudes of T 00 and T 11 are tiny. T 00 and T 11 will cause about 0.003 magnitude deviation in directivity and source match, and T 01 will cause 0.7 • phase deviation in reflection tracking. The following section will analyze how much uncertainty due to T ij results in the measured value of DUT.

IV. PROPAGATING UNCERTAINTY TO DUT
This section describes how to calculate the uncertainty of the measured DUT based on the residual model constructed in the previous section. We used a 20 dB attenuator as a 2-port DUT. The S-parameters of the DUT were measured after calibrating the VNA using the ECU. Refer to Appendix C for how to characterize the ECU and save the calibration data to internal memory.
First, we should note that all uncertainty elements are independent of each other. Namely, the residual error (δ, µ, τ ), the ECU drift (T ij ), the VNA drift (D ij ), the cable movement (CA T , CA R ), the connector repeatability (CO R ), the trace noise (N H ), the noise floor (N L ), and the VNA linearity (L) are not correlated with each other. Therefore, each uncertainty element can be isolated, as shown in Fig. 1(d). The separated uncertainty elements can be linearly propagated to the uncertainty of the DUT as follows.
where Con denote the concatenation covariance matrix for each uncertainty. For example, Con δµτ is a full two-port covariance matrix for residual δ, µ, and τ as follows; Pi δµτ means δµτ calculated by the method described in the previous section at port i. The rest of the covariance matrix can be evaluated by referring to EURAMET's VNA evaluation guide [8] or other literature. Thus J δµτ should have a form as follows; We derived each element of the Jacobian matrix for each random error in appendix B. Fig. 6 shows the measured S-parameters of the DUT and the uncertainty. We linearly propagate the covariance obtained in (7) as the covariance for magnitude and phase. After taking the square root of the diagonal matrix, it is multiplied by a coverage factor (k = 2) to obtain the  expanded uncertainty. The dotted line and blue bar represent S-parameters and expanded uncertainty (about 95% confidence interval), measured using the reference calibration kit used to characterize the ECU impedance states. The uncertainty analysis model used in this case is the VNA error model in Fig. 1(a). The solid line and red bar are the results of measurement using the ECU and the uncertainty analyzed using the residual model in Fig. 1(b). The two results agree well with each other within the uncertainty range. The proposed residual model has slightly more uncertainty than the VNA model because it includes the ECU drift T ij . Fig. 7 shows the uncertainty of S 21 in the DUT for each element. The magnitude uncertainty due to the residual error is most of the total uncertainty. However, in the case of phase, the uncertainty caused by the ECU drift T ij is considerable compared with the residual error at frequencies above 10 GHz. This clearly shows that the ECU drift T ij should be included in the uncertainty analysis of the DUT calibrated by the ECU. In other words, the stability of each impedance state of the ECU is not negligible on the DUT measurement uncertainty. Therefore, it is necessary to use an ECU whose impedance state is stable regardless of temperature.

V. CONCLUSION
In this paper, we proposed a new residual model to calculate the uncertainty of the S-parameters of the DUT when the VNA is calibrated using the ECU. The proposed method does not require raw measurements like the conventional residual model, but provides the same results as the VNA error model. Each ECU state needs to be characterized and stored in the ECU's internal memory in advance. In our case, we made a data-based model based on the reference calibration kit and then calibrated the VNA using it to have measurement traceability. Afterward, each ECU state can be measured and stored in the memory using the ''user characterization''. We also showed how to calculate the residual error directly from the uncertainty of the ECU's impedance state. In addition, the drift T ij of the ECU was evaluated and included in the uncertainty model. The measurement results showed that the VNA error model and the proposed residual model agreed well within the uncertainty. It is also shown that the effect of the drift T ij of ECU on the uncertainty of the DUT is not tiny, and should be included in the uncertainty analysis.  Fig. 8 shows a block diagram for the entire measurement process. First, we should characterize each impedance state of the ECU and evaluate its uncertainty. At this time, there is no need for a separate DUT calibration process when the characterized value is saved in the internal memory of the ECU. Please refer to Appendix C and D for this procedure. We now transform the uncertainty of the ECU impedance state into the residual error δ, µ, τ (see section III. A). And we also evaluate the temperature drift T ij of the ECU (see section III.B). Finally, other random errors (CA T , CA R , CO R , N H , N L , L, D ij ) in Fig. 1(b) are evaluated according to [8]. The ECU evaluation process is now complete. Then, the VNA is now calibrated using the ECU data (stored in internal memory) for DUT measurements. Finally, S-parameters of DUT is measured using the calibrated VNA, and the uncertainty of the DUT is evaluated (see Section IV).

APPENDIX B JACOBIAN MATRIX FOR RESIDUAL MODEL
We could separate each uncertainty element in the residual model for ECU when they are independent of each other as shown in Fig. 1(d). For a 2-port device, we can divide each uncertainty element as shown in Fig. 9. For example, when we consider the cable stability, A 11 and A 22 become CA R of port1 cable, and A 21 and A 12 become CA T . Also, B 11 and B 22 become CA R of port2 cable, and CA T can be assigned to B 21 and B 12 . Thus, the expected values of A ii and B ii are 0, and A ij and B ij are 1, respectively. The S-parameters at the port1 (a 0 , a 1 ) and the port2 (b 0 , b 1 ) are as follows Then, we take partial differentiation with respect to the real part and the imaginary part, respectively. Thus, the Jacobian matrix of (7) can be constructed using the following.
For A ij Operator * changes S ij to S ji and A ij to B ij . For example

APPENDIX C CHARACTERIZATION OF ECU STATES
In [5], each state and uncertainty of the ECU were precisely characterized using the airlines and NIST MUF. However, most secondary calibration labs employ the SOLT calibration kit. Thus we will describe a method for characterizing the ECU using the calibrated SOLT reference calibration kit. Fig. 1(a) shows a general 1-port VNA error model. The measurements and definitions of the calibration kit, and the VNA error terms e 00 , e 11 , e 10 e 01 have the following relationship.
where the subscript Meas means measurements and the superscripts O, S, L mean open, short, load reference calibration kit, respectively. When (B-1) are expressed as AE = M similar to (2), the general solution of LSQ for E is as follows.
Then, each state of the ECU can be determined.
The above process is included in the VNA as a process called ''user characterization''. In order to perform ''user characterization'', the VNA should be calibrated in advance. After the VNA is calibrated, connect the ECU to the test port cable and perform ''user characterization''. When this process is finished, the data for each impedance state of the ECU is automatically saved in the internal memory.
To have measurement traceability, the reference calibration kit applied for VNA calibration requires a ''data base model'', not a nominal value. The ''data base model'' is usually provided by the laboratory where the reference calibration kit has been calibrated. It can also be created manually from the calibration data of the reference calibration kit. If the calibrated data is in SnP or CITI format, the user can create the data base model with the calibration kit editor built into the VNA, or with a program provided separately [18]. Also, SnP or CITI files are in ASCII format, so the user can create their own.

APPENDIX D UNCERTAINTY OF ECU STATES
The VNA measurement model shown in Fig. 1 contains various random errors (cable movement CA R , CA T , connector repeatability CO R , trace noise N H , noise floor N L , linearity L, drift D ij ) that may occur during VNA measurement [7], [8], [12]. Thus, it more precisely propagates the uncertainty in the reference calibration kit to each ECU state. A detailed characterization method for random errors can be found in [8]. The reflection coefficient M at the input port is as follows [8]; 8m3(− (S 12 ) + c 1 (S 12 ) + c 2 (S 21 ) + c 3 S12 X − c 4 S12 R ) m 1 , where E 00 = e 00 + D 00 (C-2) and we can simplify the connector repeatability CO R and cable movement CA R , CA T as C ij with a linear approximation.
(C-1) can be rearranged for as follows in (C-8), as shown at the top of the page. When the uncertainty of the reference calibration kit is propagated linearly using the Taylor series, it can be expressed as the product of the Jacobian matrix and the covariance matrix as follows [14].
Jacobian matrix J REF−ECU is as follows.
where p is the total number of states of the ECU. Each element in J REF−ECU can be obtained by numerically differentiating (C-8). Also REF can be provided by the calibration lab or can be constructed from the calibration data as a form of (5). The effect of other random errors can also be propagated to the uncertainty of the ECU states using the same procedure. Jacobian matrixes for each random error are obtained by numerical differentiation of (C-8), and their covariances are propagated to the ECU states, similar to (C-9). For example, the noise floor N L can be calculated as follows.
NL is the covariance value of the noise floor evaluated as in [8], J NL is the Jacobian matrix described above, and ECU _NL is the covariance of the measured noise floor propagated to each ECU impedance state. Note that from (C-1) to (C-8), the calculation formulas are only represented at a single frequency for simplicity. However, each covariance matrix and Jacobian matrix can be easily extended to include the correlation between the cross frequencies. The total covariance for the ECU states is the sum of covariance