Statistical Behavior of PMU Measurement Errors: An Experimental Characterization

Different power system applications based on synchrophasors measured in different nodes of the electric grid require information about the statistical distribution of the errors introduced by the phasor measurement units (PMUs). The performance of these applications can be significantly affected by possible incorrect assumptions. The Gaussian distribution has been historically assumed in most of the approaches, but some more recent studies suggest the possibility of considering different distributions for more accurate modeling of the actual situation. In this article, proper statistical tools applied to the results achieved through a high-performance experimental test system are proposed to assess the statistical distribution of PMU errors under controlled steady-state conditions, thus providing a basis for defining suitable models to be used in specific applications.


I. INTRODUCTION
P HASOR measurement units (PMUs) are the key instruments for the most advanced monitoring, protection, and control applications of modern power systems [1]. PMUs were originally considered for use in transmission systems, but they are now expected to become a valuable tool also for distribution grid monitoring [2]. This new application level was driven by the technological improvement of the different elements that compose the device architecture, as well as by the availability of specific algorithms studied for dealing with the peculiarities of the voltage and current signals existing in the distribution systems [3].
Indeed, it is important to highlight that a PMU is a complex system where each element, be it hardware, as the acquisition system, or software, as the evaluation algorithms, is a source of uncertainty. The overall performance is strongly related to both the specific setup and the implemented algorithms, making PMUs very different from each other [4].
PMUs evaluate the electrical signals of voltages and currents, synchronizing the acquisition and measurement process to the UTC (Coordinated Universal Time) through time synchronization capabilities integrated into the device [5]. In particular, a PMU can include a global positioning system (GPS) receiver or, when the precision time protocol (PTP) is adopted, it can be seen as a client element in a time-synchronized packet network, where a master time source provides time synchronization. The time synchronization requirements typically considered in the synchrophasor estimation are below the microsecond [6].
The measured values provided by PMUs are the fundamental synchronized phasors, frequency, and rate of change of frequency (ROCOF). In this context, the metrics used to evaluate the performance of a device under controlled and specified conditions are the total vector error (TVE), the frequency error (FE), and the ROCOF error (RFE) [7]. Focusing on the synchrophasor evaluation index, the TVE, which is a relative vector error, is a valuable tool for summarizing the performance of a device or comparing two devices under the same test condition; however, as a single index, it does not allow us to understand the causes of the errors profoundly [4]. Indeed, the TVE encloses the magnitude, phase-angle, and synchronization errors. In particular, due to the strong bond between the phase-angle and synchronization, it is important to evaluate the synchronization quality and to discriminate only the phase-angle error (PE) introduced by the acquisition system and measurement process. For this reason, in the rest of this article, we will analyze the results in terms of magnitude errors (MEs) and PEs in the presence of monitored synchronization conditions. Different power system applications are based on the synchrophasors data provided by PMUs installed in different nodes of the electrical grid [8]. Thanks to the absolute and common time reference, all the data provided by the PMUs can be aligned and properly processed [9] to obtain the state of the grid with high accuracy. In this context, several applications relying on synchrophasor measurements, e.g., those based on state estimation, require knowledge of the statistical distribution of the error of PMU measurements. A first clear distinction has to be done between systematic and random contributions. Indeed, the systematic contributions do not change in different repeated observations and, thus, if properly identified, their effects can be compensated for [9]. On the contrary, random errors, which are introduced by the unpredictable variations in the behavior of the measurement devices, play a decisive role in determining the distribution of the measurement error to be used in state estimation procedures [10]. It is important to recall that instrument transformers can give a significant contribution, mainly systematic and depending on their accuracy class, to the errors of the whole PMU measurement chain. In this article, the focus is on PMU behavior since PMUs are expected to be applied in different contexts and it is extremely useful to have a description of each element of the measurement chain to provide further insight and models.
Historically, the errors from a PMU device are assumed to have a Gaussian distribution [11], [12], [13], [14]. It should be noted that, if an incorrect distribution is assumed for PMU measurement errors, the results obtained for a power system application might lead to incorrect actions [15]. Indeed, the estimation of the state of the power grid can deviate significantly from the actual operation due to an incorrect measurement error model [16]. Nowadays, in the scientific literature, different papers suggest using a non-Gaussian model for PMU errors in specific applications based on synchrophasor measurements. In [17], PMU MEs and PEs are statistically analyzed in order to prevent corrupted data from negatively influencing PMU-based applications. In [18] a deep neural network framework is proposed to perform distribution system state estimation for different network configurations, considering real-time PMUs and non-Gaussian noise. In [19], an ANN-based multiarea state estimator that allows mitigating the impact of outliers in the case of non-Gaussian noise is presented.
In [20], based on measurements acquired from the field, it is shown that the voltage MEs and PEs follow non-Gaussian distributions with long tails. However, the grid conditions can influence the error models. Similar results are obtained in [10], where, considering offline and online measurements, non-Gaussian distribution is also found for voltage synchrophasor errors. Moreover, Huang et al. [10] proposed the Gaussian mixture models (GMMs) for modeling the distribution of the synchrophasor measurement errors. In [21], an analysis of the PMU measurements from a calibration process is described, which considers the influence of the two classes of accuracy proposed in [7]. However, the analysis is performed considering dynamic conditions of signals as amplitude and phase modulation or frequency ramp, and the non-Gaussianity of the results can be related to the specific considered conditions. This article aims at achieving, through laboratory experiments under controlled conditions, a detailed statistical characterization of PMU voltage MEs and PEs, so that a proper PMU error model can be suggested for PMUbased applications. First, different from the above-referenced papers, where the error is evaluated from measurements acquired directly from devices installed in the field or under nonstationary conditions, here the data are related to controlled laboratory conditions where an accurately synchronized signal generator is used and the device under test is kept under steady-state conditions for a reasonable time. This allows the actual contribution of the device to be clearly separated from the possible variability of the nonstationary real signals. Then, proper statistical tools are used to process the obtained measurements and highlight similarities and differences.
In particular, three commercial PMUs from different vendors have been characterized, with the basic idea of providing general considerations device based to describe the PMU errors under steady-state conditions and controlled synchronization conditions. The reported analysis is intended as a bridge between different conclusions drawn in the literature through various approaches and tests. Fig. 1 shows the general architecture adopted to perform the experimental tests in the instrumentation and measurement laboratory of the University of Cagliari.

II. EXPERIMENTAL TEST SETUP AND DATA ANALYSIS A. TEST ARCHITECTURE
The high-accuracy power signal generator and calibrator Omicron CMC 256plus provides the synchronized threephase voltages used to characterize the device under test in different test conditions [22], [23]. Its high accuracy allows for testing a wide range of devices, including energy meters of class 0.2. The typical value of the magnitude accuracy is 0.03% in the range considered in the following tests. During the last calibration process, a maximum error below that value was reported (in particular the maximum errors are −0.007 %, −0.03 %, and 0.002 % for phases A, B, and C, respectively). The time synchronization is ensured by an external GPS receiver (Meinberg Lantime M1000) as the primary high-accuracy time source. Then, time synchronization is provided to the calibrator through PTP, considering the "power utility" profile [24], [25]. During synchronization issues, if the time synchronization error exceeds 1 μs between the primary time source and the calibrator, the device automatically stops the signal generation to prevent inconsistencies in data due to an undisciplined clock. The calibrator is managed by the software suite "Test Universe," which includes a specific module to verify electric devices under steady-state and dynamic conditions [26].
As far as the synchronization of the PMUs under test is concerned, when the device is not equipped with an internal GPS receiver, the time synchronization is again provided by the above-mentioned external GPS receiver, so that the required synchronization conditions are achieved.
The calibrator provides the same reference voltage simultaneously to all PMUs. The PMUs are configured to take and report the measurements in the IEEE C37.118.2 data packets [27]. PMUs can be configured to either report only the positive sequence of the monitored quantities, thus reducing data packet size, or transmit all data relating to the three phases, prompting the phasor data concentrator (PDC) to evaluate the positive sequence if needed. In [21], the results of the statistical analysis for the positive sequence voltage synchrophasor are reported, but to investigate more deeply the device behavior, it is better to study the three signals separately, so that the behavior of the individual acquisition channels can be captured and emphasized. Since the three sensing channels of the PMU are independent in terms of conditioning and analog-to-digital conversion, the analysis in the following considers the results for all three voltage phases. As for the data transmission, the outputs of the PMUs are connected to a development computer with PDC functionality, developed in the LabVIEW environment to receive and store the TCP data packets encapsulating the payload in the format described in [27]. The PDC-enabled computer in Fig. 1 is also used to configure and control the power signal generator by sending settings and commands in a Test Universe format.
The data collection process starts 30 min after the warmup of the devices under test. In this way, possible variations during the starting time are not considered in the subsequent statistical analysis. Then, the PDC stores 1 h of data, so that the stability of the devices during an extended test under nominal conditions can be verified. The PDC also checks the state of the PMU synchronization throughout the duration of the test. The tests are performed under a controlled temperature of about 23 • C.

B. STATISTICAL ANALYSIS
The focus of the analysis is on providing the behavior of measurement errors of commercial PMUs under steady-state conditions. The data collected by the PDC are evaluated in terms of MEs and PEs with regard to the reference values set in the calibrator. The TVE index is defined in [7] as The TVE is reported in a series of discrete values n (corresponding to the reporting instants) and X r (n) and X i (n) are the real and imaginary parts of the measured synchrophasor. X r (n) and X i (n) are the real and imaginary parts of the reference synchrophasor configured in and generated by the power signal generator. The TVE in (1) can be reported in terms of relative ME and PE as where ME and PE can be expressed as where atan2 indicates the four-quadrant inverse tangent and the phase-angle difference is considered as wrapped in (−π, π). The analysis is performed considering different steps and is divided into three parts depending also on the statistical tool considered to analyze the data, as described in the following.
1) Autocorrelation Analysis: To find a statistical representation of PMU errors, it is important to consider a set of uncorrelated measurements when possible. In this context, the AEs and PEs are evaluated considering the autocorrelation function (ACF) along the time series of measured data in order to estimate and then reduce the self-similarity among the samples. The autocorrelation analysis can also help in uncovering hidden patterns in our data. For example, the synchronization process is commonly used to discipline the acquisition process of the different input channels [28], [29]. In this way, dynamics in the synchronization system can influence the corresponding acquisition process [30]. These considerations are strictly dependent on the manufacturing characteristics of each PMU. Unlike the other scientific works discussed in Section I, this procedure is taken into account here for the first time, mainly  to point out possible misleading results due to strong autocorrelation in the data. 2) Gaussianity Test: It is performed with the Shapiro-Wilk [31] test and with the D'Agostino test of skewness [32]. The Shapiro-Wilk test is one of the most widely recognized tests in the scientific literature for determining whether or not a data sample is from a normal distribution. Under the hypothesis of Gaussianity, the set of analyzed measurement errors should be symmetrical and D'Agostino test is considered a valid instrument to detect a significant asymmetry of data. In order to check the hypothesis of normal distribution, a confidence level of 95 % is considered in this article. In this way, the significance level of the test results, α, is set to 0.05. The p-value represents the test result to be compared against α, as in [20]. In the Shapiro-Wilk test, with the above assumption, the Gaussianity hypothesis can be rejected when p is < 0.05. When p ≥ 0.05, the null hypothesis cannot be rejected, which means that the AE or PE distribution does not differ significantly from the normal distribution. For the D'Agostino test, when p ≥ 0.05, there is no reason to reject the null hypothesis that the data distribution is symmetrical. In other words, when the two tests are passed, we can assume normality as a reasonable model. The combination of the two tests is a powerful tool for analyzing the collected data and the second one is used here as a confirmation of the first one. Moreover, in the following, this analysis is accompanied by the graphical representation of the data, considering the histograms of the errors to allow an immediate visual inspection. 3) Q-Q Plot: To accompany the Gaussianity tests, Q-Q plots are also used to analyze measurements. The Q-Q plot sorts data in ascending order, and then plots them against the quantiles calculated from a theoretical distribution. The number of quantiles is chosen to match the sample data size. This graphical representation allows visual analysis of the different behaviors of the measurement errors of the considered PMUs. The RStudio environment [33] based on R software for statistical computing and graphics has been used to process the data for all the tests discussed in the following.

III. TESTS AND RESULTS
The outcome of the analysis of the three-phase voltage MEs and PEs of three commercial PMUs (referred to as PMUs 1, 2, and 3) is reported in the following. The three considered PMUs are different stand-alone devices from different manufacturers, with their own hardware and software equipment, and are configured for M-class accuracy and reporting rate equal to 50 frames/s. The common test signals have a root mean square magnitude of 50 V at the nominal system frequency of 50 Hz.

Figs. 2 and 3 show the results of the ACFs for MEs and
PEs from the three PMUs under test (only phase C of each device is shown here for brevity) considering a maximum time lag of 8 s, corresponding to 400 measurements. The assumption of stationarity is adopted.
It can be seen that the autocorrelation follows different behaviors for MEs and PEs within the considered time interval. More in detail, from Figs. 2 and 3, it is possible to observe three different behaviors for the three PMUs. In particular, for PMU 1 Fig. 2(a), the ACF of MEs quickly decreases to negligible values. On the contrary, the PE ACF in Fig. 3(a) decreases slowly, with local maxima that appear at time lags close to the multiples of 50 measurements, i.e., distances that correspond to multiples of 1 s. For PMU 2, the ACF of MEs in Fig. 2(b) is described by a quasi-constant autocorrelation value lower than 0.2, while the ACF function of the PE in Fig. 3(b) is characterized by a similar behavior as PMU 1, i.e., by the presence of periodical local maxima. PMU 3 is characterized by an ACF of MEs that falls off quickly after a few samples, but periodically shows peaks again, even if low (< 0.2), which are distributed in the considered time window [see Fig. 2(c)]. The ACF of the PE in Fig. 3(c) is similar to that of the other PMUs, but with low-correlation time lags and high-correlation time lags alternated in a more distinct way.
The results provided by the ACF thus highlight two different behaviors for the two measured quantities. For the magnitude, the results, even if different between PMUs, show a significant decrease in ACF, i.e., an evident decorrelation of the measurements after a few reporting intervals.
For the PEs, instead, the three ACF functions indicate a higher correlation with time, particularly at data shifts multiples of the reporting rate value. The three PMUs show significantly different phase-angle ACF patterns, but they all reveal that the correlation between measurements depends mainly on the timebase characteristics and on the synchronization strategy. In fact, the periodical and repeated presence of higher values can be related to the pace and operating mechanism of time synchronization, which is generally updated every 1 s, thus periodically adjusting the offset and influencing the PMU timebase error that impacts directly on the evaluation of phase-angle. PE strongly depends on how the time synchronization is implemented within the acquisition process and on the intrinsic characteristics of the instrument clock because every clock has a different drift and inertia. Furthermore, several tests performed using different synchronization sources have confirmed the link between the found autocorrelation signature and each PMU characteristic.
The manifold behaviors revealed by the ACF analysis for the different devices emphasize the importance of investigating the time dependence of the data series, which can lead to many different models when implementing applications. Time dependency analysis is also preliminary to any assessment of statistical properties (e.g., of Gaussianity), in order to decrease the impact of the autocorrelation effects on the obtained results. Therefore, here the ACF outcomes are used to identify a suitable time lag that reduces autocorrelation and, thus, to find a downsampling factor of the error series that allows dealing with an almost uncorrelated vector of MEs and PEs before proceeding to the Gaussianity tests. The original time-aligned vectors of MEs and PEs are decimated, considering a reasonable value with a low value of ACF for each system phase and each PMU. The new error vectors are characterized by a different size depending on the downsampling factor and are considered in the following analysis.

B. GAUSSIANITY TESTS
In the following, the well-known Shapiro-Wilk test is considered to evaluate the hypothesis of Gaussianity for each vector of MEs and PEs. In support of the previous test, the explicit detection of skewness in the data is evaluated by the D'Agostino test. In this context, it is essential to recall that the test is sensitive to the sample size. Therefore, if the vector contains few data, it may often pass the Gaussianity tests. On the contrary, with huge vectors, outliers can easily lead to null hypothesis rejection. For these reasons, the Shapiro-Wilk tests are here performed using vectors of different sizes and considering the suggested limits of the test. Indeed, RStudio limits the sample size in the range from 3 to 5000 samples to prevent misleading usage. Furthermore, the Gaussianity test is accompanied by the graphical representation of the histograms of magnitude and PEs for a visual inspection (based on 1000 samples). Table 1 shows the results of the Gaussianity test evaluated with the Shapiro-Wilk and D'Agostino test for the quasi-uncorrelated MEs and PEs of the three PMUs for the three system phases and considering four different vector sizes, including 100, 500, 1000, and 3000 measurements, respectively. In order to maintain the possibility of comparing the results with previous work (e.g., [20]), the results are reported for both tests in terms of p-value, while it is set α = 0.05 as mentioned above. In this context, the Gaussianity can be considered likely if p ≥ 0.05 for each test. In summary, PMU 1 passes the Gaussianity tests for all considered datasets and sample sizes. The MEs of PMU 2 are non-Gaussian only for phase A, while they can be modeled as Gaussian for phases B and C. Moreover, the PE passes the Gaussianity tests for all three phases. Considering PMU 3, MEs pass the Gaussianity tests for all the phases and for the four considered sizes, while the PEs cannot be considered as Gaussian under the results of the Shapiro-Wilk test.
In this scenario, the MEs of phase A in PMU 2 and all the three PEs in PMU 3 require an in-depth analysis using also graphical tools. Fig. 4 shows the decimated errors in a 60-s window (3000 measurement errors) considered for the Gaussianity tests. In particular, Fig. 4(a) shows the ME for PMU 2 phase A, where it is possible to observe that the values are mainly concentrated at the bottom of the figure and only a few measurements can be associated with larger errors, thus introducing a certain degree of asymmetry in the distribution. Furthermore, the dataset was controlled over different time windows, which confirmed that this behavior

FIGURE 4. (a) ME for PMU 2 phase A and (b) PE for PMU 3 phase C over 3000 samples.
is due to the specific behavior of phase A in PMU 2, and is not present in the other channels. This peculiar behavior, whose origin is difficult to assess, highlights once more the importance of analyzing PMU channels individually to find possible differences and have fine-tuned models. Additional tests have been performed also swapping generator channels, always confirming the specificity of the first channel of PMU 2. Fig. 4(b) shows the PE trend for PMU 3 phase C. Similar behavior can be found in all three phases. It is possible to observe that the errors are mainly distributed at three different levels. This particular behavior, evaluated in different time windows and tests, is intrinsically dependent on the instrument (probably its synchronization stage) and confirms the non-Gaussian distribution of PEs for PMU 3. Fig. 5 reports also the histograms concerning the MEs of phase C, respectively for PMUs 1, 2, and 3, evaluated over 1000 decimated samples as discussed before. Fig. 6 shows the corresponding histograms for the PE. PMU 1 is characterized by relative MEs and PEs with a Gaussian distribution, which is also confirmed by the shape of the histograms in Figs. 5(a) and 6(a). Figs. 5(b) and 6(b) are related to the MEs and PEs of PMU 2, and they appear to confirm the validity of the previously found Gaussianity. The same results can be obtained for phase B and only for PEs of phase A. PMU 3 features the lowest relative ME [see   almost "discrete" behavior of these errors and clearly demonstrates their non-Gaussianity in Fig. 6(c). The same types of random error distributions can be found (within the generator uncertainty) also exchanging the generator channels and shifting the phases opportunely, thus corroborating the above findings.

C. Q-Q PLOT ANALYSIS
The analysis of Q-Q plots, here performed over a record length of 1000 measurements, increases the understanding of the results reported in Table 1 and allows evaluating where the assumption of Gaussianity is violated. For all the collected data, the normalized quantiles are set in the range [−2.5, 2.5].
Figs. 7 and 8 represent the Q-Q plots for MEs and PEs, respectively, for phase C of all the considered PMUs. From Figs. 7(a) and 8(a), it is possible to immediately confirm the results reported in Table 1, where PMU 1 passes the Gaussianity tests for the three considered sets of data. In the same way, for PMU 2, whose results are represented in Figs. 7(b) and 8(b), it can be clearly observed that the hypothesis of Gaussianity is again corroborated by the normalized quantiles. Fig. 7(c) confirms the results of Table 1 for MEs of PMU 3, which falls in the Gaussianity hypothesis. Nevertheless, the same distribution cannot be chosen for the PEs of Fig. 8(c), whose distribution is strongly non-Gaussian, as clearly visible from the three abrupt transitions in the Q-Q plot corresponding to the three peaks that can be recognized in Fig. 6(c). Once again, PEs are more prone to variability and intrinsic mechanisms that, when the errors are low as those found here, lead to non-Gaussian behavior, mainly because they strongly depend on the synchronization system operation as discussed above. For instance, periodical adjustments and oscillations typical of clock synchronization can result in more probable values at the distribution sides, as in Fig. 8(c).

IV. CONCLUSION
The presented work aimed at statistically evaluating the voltage MEs and PEs of PMUs under steady-state conditions in order to discuss the recent literature findings from a metrologically sound perspective. The first step was to evaluate the autocorrelation of the measurements, thus showing that particularly PEs can be strongly correlated and models found on original data without keeping correlation into account can be misleading. This approach showed to play an important role in evaluating the Gaussinity hypothesis. The performed analysis highlights that Gaussian distribution is often a meaningful choice, especially for MEs. The deviation from Gaussianity can emerge, depending on the specific device, measured quantity, and considered channel.
In this regard, the proposed analysis can be seen as a useful step to derive well-suited and application-oriented models. The outcomes of the reported analysis are intended to act as a bridge among different conclusions drawn in the recent literature through various approaches and tests. In particular, the statistical distribution of errors was here analyzed through laboratory experiments under controlled conditions so that it is possible to discriminate stationary models from nonstationary effects. This prevents misinterpreting error distributions obtained from raw measurement data from the field. The results obtained testing three commercial PMUs indicate that a general model of PMU random error is impossible to provide under common conditions. However, assuming that the monitored network is composed of homogeneous devices from the same vendor, after characterization of the errors in the laboratory environment, an error model can be derived which is tailored to the needs of the applications in the considered network.