Positioning Performance Comparison Between GPS and BDS With Data Recorded at Four MGEX Stations

In December 2018, the BeiDou III (BDS-3) basic system was completed and began to provide global services. along with the BeiDou second-generation satellite navigation system (BDS-2) and Beidou first-generation satellite navigation system (BDS-1), the whole BDS has advantages in the number of satellites, coverage, and positioning performance. This paper focuses on the comprehensive evaluation of BDS by comparing the data quality and positioning performance of BDS and those of Global Positioning System (GPS). The performance evaluation and comparison is based on 7-day observation data collected in April 2020 at three different geographical locations in China. The experimental results show that GPS and BDS have a number of significant differences. The number of visible satellites in BDS is twice that of GPS in China. The SNRs of GPS signals of different frequencies are quite different from each other, while BDS signals of different frequencies have very similar SNR observations. BDS-3 and GPS achieve similar positioning performance in terms of internal positional error, external positional accuracy, coordinate time series and Dilution of Precision (DOP). The mean and STD of the internal positional error of GPS and BDS are less than 3 m, while the external positional accuracy of both systems is better than 5 m. It is also observed that the overall variation in coordinate time series of the three coordinates of BDS is slightly smaller than that of GPS. In addition, the BDS DOP is basically equivalent to the GPS DOP, although occasionally the former is greater than the latter. The results are largely contrary to most of the results associated with performance comparison between BDS and GPS reported in the literature. The main reason would be that the number of visible satellites, observations and DOP of BDS are superior to GPS and single BDS-1, BDS-2 or BDS-3 in China. We believe that the development of BDS is still in progress and BDS is making continuous performance enhancement.

system providing positioning services after GPS in the United States and GLObal Navigation Satellite System (GLONASS) in Russia [2]- [4]. A total of 19 BeiDou third-generation satellite navigation system (BDS-3) satellites had been on orbit to complete a preliminary system for global services by November 19, 2018. BDS-3 adopts the constellation of 3 geostationary orbit satellite (GEO), 3 inclined geosynchronous orbit (IGSO) and 24 medium-Earth orbit (MEO). The satellites have communication capabilities and can operate autonomously without the support of ground stations. It is expected that the last networking BeiDou satellite has launched in June 23, 2020. By 2035, BDS will build a more ubiquitous, integrated and intelligent space-time system [3].
The Signal-In-Space Range Errors (SISRE) directly affects the positioning accuracy and the reliability of the system. Previous studies, such as that by Guo et al. [5], performed the accuracy analysis of different types of broadcasting ephemeris orbits and clocks of GPS satellites. The accuracy of different types of satellites is mainly reflected in its stability. BDS-2 has been built at the end of 2012, and Zhang et al. [6] calculated the BDS-2 SISRE, confirming that it has low correlation in the radial-clock difference and tangential-normal, with the existence of micro-correlations among the other components. Pan and Cai [7] concluded that the accuracy of BDS-2 broadcast ephemeris is lower than that of GPS. Jia et al. [8] showed that the accuracy of the non-GEO satellite broadcast ephemeris in BDS-2 is better than that of GEO, and the radial accuracy of BDS-2 satellite broadcast ephemeris is the best. For the GNSS SISRE, Montenbruck et al. [9] confirmed that the SISRE of GPS and Quasi-Zenith Satellite System (QZSS) are smaller than those of the GLONASS and BDS. Yang et al. [10] mainly analysed the performance characteristics of the demonstration system of BDS-3, indicating that BDS-3 achieves considerable performance improvement compared to BDS-2. Wang et al. [11] proposed an improved method for calculating BDS-3 Dilution-of-Precision (DOP) using several constellation parameters without using real-time or simulated model ephemeris.
Previous literature focused on the positioning performance comparison between GPS and BDS-2. The positional accuracy of GPS and BDS-2 static precise point positioning (PPP) reaches several centimetres to millimetres, and dynamic PPP accuracy reaches decimetres to centimetres [12]- [15]. Liu and Ye [12] directly used the International Global Navigation Satellite System Service (IGS) tracking stations to calculate the orbital parameters. Their results showed that the accuracy in the X, Y, and Z directions was better than 20 cm compared with the case where the satellite orbit is calculated by the IGS tracking station. Yi et al. [14] proposed a method for real-time PPP using the regional GPS continuous operation reference station (CORS) and the IGS ultra rapid (IGU) released by IGS. This method can be used for PPP within a range of hundreds of kilometres from the CORS area. Santerre et al. [16] conducted a study using GPS+GLONASS+BDS that showed an improvement of approximately 20% in horizontal coordinates and of nearly 50% in the vertical component compared to the GPS+GLONASS at obstructed sites. With the completion of the BDS-2 in 2012, Yang et al. [17] conducted a preliminary evaluation of the positioning performances of BDS-2. They concluded that the accuracy of BDS-2 single point position (SPP) is much inferior to GPS, but meets the BDS-2 design requirements. Li et al. [18] used the combination of all the four GNSS constellations (GPS, GLONASS, BDS, and GALILEO) to perform real-time PPP and confirmed that the combined system has a faster convergence speed, better accuracy, and higher reliability than a single system based scheme [19]- [21]. To improve the utilization of raw observation, Liu et al. [22] directly used the raw observations to perform multi-GNSS SPP. Their preliminary results showed that this method can improve both the static and dynamic SPP accuracy to a certain extent. Zhang et al. [23] investigated the positioning performance of BDS-3 and demonstrated that real-time kinematic (RTK) and PPP achieve the cm-level accuracy based on the latest BDS-3 data.
Although the positioning performance of single BDS-2 has been investigated extensively, there are also few literatures to analyze the positioning performance of BDS-3 in recent years, there are only a few reports on the performance evaluation of whole BDS (i.e. BDS-1+BDS-2+BDS-3). With the completion and gradual improvement of BDS, timely and accurate analysis of the positioning performance of BDS is of great significance for BDS performance enhancement and future BDS modernization. Although modernized GPS and BDS provide dual-frequency or triple-frequency observations, these data are not available in some epochs. Therefore, in this paper, by using GPS and BDS singlefrequency observations collected from four MGEX stations in Wuhan, Urumqi and Lasha in China, we first evaluate the observation data quality based on satellite skyplot, satellite visible number, observation time and Signal-to-Noise Ratio (SNR). We then further evaluate the coordinate time series, internal positional error and external positional accuracy and DOP value of GPS and BDS. These should have certain reference significance for timely understanding of the latest BDS performance.  [3]. The relationship between GPST and BDT follows [3]:

II. PRINCIPLE AND METHOD OF POSITIONING
GPS and BDS use WGS84 and CGCS2000 as their coordinate systems, respectively, while precision ephemeris uses International Terrestrial Reference Frame 2014 (ITRF14). The definitions of the origin, scale and orientation of the WGS84, CGCS2000 and ITRF14 are almost the same. In addition, the reference ellipsoids of the three systems are very similar, with only a slight difference in the flatness. This has a very small influence on the positioning accuracy for the coordinate conversion among the different systems due to the negligible difference in the flatness of the reference ellipsoids of the three coordinate systems [24].

B. CALCULATION OF THE AZIMUTH AND ELEVATION OF THE SATELLITE
Establish a coordinate system by setting the station position p as the origin, the normal direction of the reference ellipsoid through point p is U axis, the meridian direction of p as the N axis, and the E axis as orthogonal to the N axis and the U axis, and hence the N, E, and U constitute the left-handed rectangular coordinate system, which belongs to the normal topocentric coordinate system. If the geodetic longitude and latitude of the point p is λ p , ϕ p T , the conversion relationship between the normal topocentric coordinate system of the j-th satellite and the corresponding geocentric rectangular coordinate is given by [24], where N j = N j , E j , U j T is the normal topocentric coordinate vector of the j-th satellite; X p = X p , Y p , Z p T and Y j = X j , Y j , Z j T are the geocentric rectangular coordinate vectors of p and j, respectively; R p is defined as According to Eq. (3), the distance ρ j p from station p to j-th satellite, the azimuth angle alpha j and elevation angle beta j of the j-th satellite are given by

C. ERROR EQUATION AND ACCURACY EVALUATION
Pseudo-range single-point positioning (SPP) uses satellite orbits and clock provided by satellite broadcast ephemeris and pseudo-range observations for position determination. The coordinates of a single point are determined by the spatial distance intersection method when using a single receiver which measures pseudoranges to four or more satellites. Due to the influence of atmospheric delays such as the troposphere and ionosphere during the propagation of the signal, as well as considering the clock errors caused by the satellite clock error and the receiver clock error, the pseudo-range observation equation from station p to j-th satellite are: where x p and x j are the geocentric rectangular coordinates vector of station p and j-th satellite, are the receiver and satellite hardware code delays and the multipath-induced delay, respectively.
Assume that the pseudorange measurement noise related to the j-th satellite is ε j , which is a zero mean white Gaussian variable with the variance σ 2 j . That is where σ 0 is unit weight error, which is generally around 1 m for pseudorange observations. P j is the weight of the j-th satellite and is determined by the well-known elevation angle weighting method [24]. For the j-th satellite observation, the weight is given by Assume that n satellites are observed simultaneously for a given epoch at the station p. When there are at least 4 satellites (i.e. n ≥4), the observation equation and the weighted leastsquares solution are given by [24] where V is the observational residual vector,x, x 0 p and dx are the estimated values, approximate values and correction values of coordinate vector x p , P is the diagonal weight matrix of n pseudorange, B is the design matrix, and l is the known constant vector. These vectors and matrices are given respectively by (12) and (13), as shown at the bottom of the next page. where are the approximate geocentric rectangular coordinates and their corrections of Then, according to the principle of indirect adjustment and the Eq. (11), the variance of the estimated coordinate vector x is Then the Root Mean Square Error (RMSE) of the k-th unknown parameter inx is obtained as

D. DOP CALCULATION
The design matrix B can be converted into the normal topocentric local coordinate system represented by the azimuth angles alpha and the elevation angles beta of the satellites in (15), as shown at the bottom of the next page. From Eq. (15), positioning accuracy is not only related to the observation noise standard deviation σ , affected by various error sources, but also to the geometric distribution of the satellites relative to the station, the so-called geometric dilution of precision (DOP). DOP is a concept introduced to characterize the influence of satellite geometry on the positioning accuracy, which can be described by [24]: denotes the cofactor matrix for parameters k and l. According to Eq. (17), the formula for calculating 3D Positional Dilution of Precision (PDOP) and Geometrical Dilution of Precision (GDOP) can be obtained as follows According to the error propagation law, the cofactor matrix in the normal topocentric local coordinate system is given by In addition, the Horizontal Dilution of Precision (HDOP) and Vertical Dilution of Precision (VDOP) are defined as

III. PERFORMANCE ANALYSIS
A. EXPERIMENTAL DATA AND DATA PROCESSING 1) EXPERIMENTAL DATA The data used for the study were recorded by four MGEX stations, two stations in Wuhan (JFNG, WUH2), one station in Lasha (LHAZ) and one station in Urumqi (URUM). The location distribution of these stations is shown in Fig. 1. The observation period of these four stations is from April 5th to 11th, 2020, and the sampling interval is 30 seconds. The daily dataset contains 2880 epochs of GPS and BDS.

2) DATA PROCESSING
The raw data of the pseudorange observation are processed with the SPP program as follows. (1) The observation data of the 7 days are downloaded free from MGEX website. Considering that there are relatively few observations containing both dual and triple frequencies, we used the single frequency pseudo-range observations. The types and frequencies of pseudo-range observations used at each station are shown in Table 1. The single frequency of GPS is f 1 = 1575.42 MHz, while the single frequency of BDS is f 1 = 1561.098 MHz. Note that C1C represents the pseudorange observations of the C code of f 1 on the C channel [25]. C2I represents the pseudorange observations of the C code of f 1 on the I channel.
(2) In the global ephemeris file, GPS gives satellite orbit parameters, satellite clock parameters, satellite status information and other information every 2 hours, while BDS is 1 hour. The satellite position and clock error in SPP are calculated from 7 global broadcast ephemeris files downloaded free from the MGEX website. The satellite clock error calculated using the global broadcast ephemeris has absorbed the internal delay of the satellite signal. Therefore, the internal delay of the satellite signal cannot be corrected during the SPP calculation.
(3) The ionosphere delay uses the klobuchar model (GPS K8 model) based on single-frequency GPS and BDS observation data. The eight parameters in this model are obtained by using global GNSS dual-frequency observation data. This parameter is updated daily, and its calculation model is as follows: where c is speed of light, IPP is ionospheric pierce point. The values of A IPP and X IPP and F are: where t IPP is the local time (seconds) of the puncture point IPP, (α 0 , α 1 , α 2 , α 3 , β 0 , β 1 , β 2 , β 3 ) are the 8 parameters given by the global ionosphere, φ IPP is the geomagnetic latitude of the puncture point, and the formula of φ IPP is where (λ IPP , ϕ IPP ) are the latitude and longitude under the satellite at the IPP. To learn more about this klobuchar model, please refer to [27].
(4) The troposphere delay is corrected by the Saastamoinen model [26]: where Pre p is the atmospheric pressure in millibar, T p is the temperature in Kelvin, and e p is the partial pressure of water vapor in millibar. The meteorological parameters in the model are based on standard meteorological parameters [24]. That is, Pre p = 1013.25 millibar, T p = 273.16 kelvin, and e p = 0 millibar. (5) The weight matrix P of the single frequency observation is determined by the well-known elevation angle method [24], that refers to equation (10). (6) The influence of errors (e.g. ephemeris error, Earth's solid tide and others) on the positioning accuracy is ignored. (7) The clock error of the receiver for each epoch is solved as an unknown parameter, absorbing the hardware delay of the receiver.      is mainly distributed between north to 15 degree and south to 90 degree (URUM station is north to 0 degree, and south to 90 degree). The GPS satellite coverage is wide, and the satellite orbit is concentrated and evenly distributed.

B. COMPARISONS OF SOLUTIONS
(b) The azimuth and elevation angles of BDS satellites are mainly distributed in the range of 30-330 degrees. And north to 15 degree and south to 90 degree (URUM station is north to 0 degree, and south to 90 degree), respectively. The satellite sky distribution of BDS satellites is also evenly distributed, and the satellite coverage is wide too. However, the BDS satellites occasionally cannot be observed by some stations probably due to interruptions over short periods.

b: SATELLITE NUMBER, SATELLITE VISIBILITY, AND OBSERVATION TIME
Figs. 6-9 show the sequence of observed satellites for GPS and BDS over 7 days. Table 2 shows the statistics on the   Table 2, Both systems have more than four visible satellites, the number of BDS visible satellites at the four stations is generally significantly higher than GPS satellites. The minimum and average observed number of  GPS satellites are 9.01 and 9.06, respectively, while the corresponding numbers for BDS are 14.01 and 17.11, respectively. Among them, the average number of visible satellites of WUH2 exceeds 20. It can be seen that the number of visible satellites is mainly related to the type of receiver used to receive satellite signals.
(b) As observed from Tables 3 and 4

c: SATELLITE ELEVATION ANGLE AND SNR
SNR can reflect the influence of the noise from observation files. The elevation angle of the satellite has a high correlation with the atmospheric delay on the signal propagation path, and the SNR has a high correlation with the error occurred near the station (such as multipath error). Therefore, a stochastic model with the elevation angle or SNR is used in GNSS positioning to characterize the accuracy of the observations and their correlation [26]. Theoretically, a higher satellite elevation angle or higher SNR corresponds to better observation quality. Fig. 8 shows the time series of the SNR of different frequency observations at each epoch of four stations. Table 5-6 summarize the receiver types and the average of SNR at the 4 stations, where '−' means SNR is 0 or no SNR value. After averaging the SNR value that is not 0 of the same satellite, and then averaging the SNR corresponding to the same elevation angle, Fig. 9 shows the relationship between the SNR and the elevation angle. A number of conclusions can be drawn as follows.
(a) The GPS SNR and BDS SNR of the four stations in China have almost the same magnitude. The SNR values for most observations range from 10 to 60 dB. As can be seen from Fig. 9, with greater SNR values obtained for higher satellite elevation angles, the overall SNR of satellite observations at different frequencies is proportional to the elevation angle. (b) Excluding the satellite observations that SNR is 0, the SNR of the same frequency at each station is averaged to find that the maximum GPS SNR is 49.32 dB and the minimum is 39.64 dB, while the maximum BDS SNR is 43.99 dB and the minimum value is 43.81 dB. Compared with BDS, the SNR values at two different GPS frequencies are    quite different, but the SNR values at two BDS frequencies are almost the same from Fig. 8 and Table 6.
(c) It can also be observed from the Table 5 and Table 6 that different receiver types correspond to different SNRs of the two systems, reflecting the correlation between the SNR and the receiver type [28]. The main reason for this phenomenon is that the SNR itself has a certain relationship with the hardware equipment.   (c) The variations in the N and E directions are significantly smaller than that in the U direction for both GPS and BDS. Therefore, the U direction is much more affected by observation noise.

b: COMPARISON OF RMSE
In order to check the internal positional error of GPS and BDS at the four stations, the RMSE for each epoch is calculated using Eq. (15). We put the RMSEX, RMSEY and RMESZ of the four stations together and count the RMSEs distributions for the four stations. As shown in Fig. 12 and Fig. 13.      (b) The STD and mean of RMSE in GPS is roughly within 3 m, while the RMSE in BDS is basically the same, with a deviation of about 3 m. The RMSE mean and STD in the X, Y, Z directions of the four stations are different, indicating that the accuracy of the four stations and their three coordinate components are different. This is mainly due to the different satellites and observations of each epoch.
(c) According to the comprehensive judgment of Mean and STD of the four stations, the overall internal accuracy of GPS and BDS is basically the same. Even BDS accuracy is better than GPS in some epochs or station. Although the current BDS accuracy is still meter-level accuracy, it is better than 3 m, which is consistent with the officially released positioning performance [29].

c: COMPARISON USING PRECISION POINT POSITIONING (PPP)
To compare the external accuracy, this study uses the PPP FIX module in RTKLIB to calculate the positions of these four stations. The average value of the static PPP calculated by the last multiple epochs every day is regarded as the ground-truth positions. The implementation of PPP is realized as shown in Table 11.
Figs. 14-15 show the X, Y, Z three components time series of the difference between SPP-based and PPP-based position estimates for GPS and BDS at the four selected stations. Tables 12 and 13 give the mean of the absolute value of the difference between SPP and PPP and the STD statistics of the difference for GPS and BDS.
Based on Figs. 14-15 and Tables 12-13, we can draw the following conclusions.     At some stations, the accuracy and stability of BDS is better than GPS.
(c) The large difference between the accuracy values in the X, Y, Z directions of GPS and BDS is mainly due to the large difference between the orbit errors of the broadcast ephemeris in the radial, tangential and normal direction [18], especially the difference in the normal direction is the largest, resulting in the largest difference in the Y direction.     Table 14.
From the results, we can then conclude: (a) The mean and STD of GPS GDOP values of the four stations are 1.96-2.95 and 0.19∼0.73, while those for BDS are 1.96∼2.60 and 0.19∼0.40. BDS DOP is slightly smaller than GPS, indicating that BDS has a better spatial distribution than GPS in china.
(b) The HDOP in the horizontal direction of GPS and BDS is smaller than the VDOP in the vertical direction. BDS DOP is on average very similar to the GPS DOP in China.

IV. CONCLUSION
In this study the mathematical model of SPP and the calculation formulas of DOP with satellite elevation and azimuth angles are derived in detail. The observation qualities of GPS and BDS are compared using skyplots, observation time and SNR of the observation satellites. The positioning performance of the two systems is compared using the time series of N, E, and U coordinates, the internal and external accuracy and DOP. Our main conclusions are as follows.
(1) the distribution of the BDS satellites is close to that of the GPS satellites. The azimuth of the BDS satellite is mainly distributed at 30-330 degrees in China, and the elevation angle is mainly concentrated between north to 15 degree and south to 90 degree. Therefore, The BDS satellites are evenly distributed and have a wide range of satellite coverage. However, compared to GPS, occasionally a few BDS satellites have interruptions and short observation time.
(2) The number and observation time of BDS satellites are better than those of GPS in China. The number of BDS satellites has reached 47 (7 GEO+11 IGSO+29 MEO). GPS and BDS reach an average of 9.01 and 23.79 visible satellites in China. Moreover, the average observation time of GPS satellites is relatively uniform, with an average of 7 hours. The observation times of the BDS's GEO, IGSO, and MEO satellites are approximately 23 hours, 18 hours, and 7 hours, respectively. This shows that the observation periods of BDS MEO satellites are close to those of the GPS satellites. In addition, the SNR values of GPS and BDS range from 10 to 60 dB. The SNRs of the three frequencies of BDS are almost the same, while those of GPS dual frequencies are quite different.
(3) The variations of the BDS coordinate time series in N, E, U directions as a whole are slightly less than those of GPS. The average variations in the three directions for GPS are 0.94, 1.49, and 4.30 m respectively, while those for BDS are 0.41, 0.48 and 1.31 m. The N, E direction variations in GPS and BDS coordinate time series are smaller than those in the U direction, particularly due to the high observation noise in the U direction.
(4) BDS has higher positioning accuracy and better stability than GPS regardless of the internal positional error and external positional accuracy at some stations in China. The internal accuracy of both GPS and BDS is approximately 3 m. Comparing with the external positional accuracy between GPS and BDS, we show that the external accuracy of GPS and BDS is better than 5.0 m, and the BDS accuracy is more uniform and stable compared with that of GPS. This is mainly due to the fact that the number of visible satellites, observations and DOP of BDS are superior to GPS in China. (5) The estimated DOP value of GPS and BDS is no more than 3. The BDS DOP is smaller than GPS DOP in China in most epochs. The DOP in BDS at individual epoch is greater than that of GPS. But in some cases, the BDS DOP is smaller than that of GPS. Finally, for both systems the accuracy in the horizontal direction is higher than that in the vertical direction.
(6) It has been confirmed that the positioning performance of BDS in China is basically the same as that of GPS, and some aspects are better than GPS. Note that this paper only evaluates the positioning performance of the two system with data observed in several regions of China. Thus it is useful to do more performance comparison with data recorded in other regions of the world.