Automated RPCs Bias Compensation for KOMPSAT Imagery Using Orthoimage GCP Chips in Korea

High-resolution satellite images, including KOMPSAT, WorldView-3, and Pléiades, are widely used for mapping and environmental monitoring. In particular, satellite images with high spatial resolution have accurate location information and provide essential spatial information. Most high-resolution satellite image files are provided with rational polynomial coefficients (RPCs) that can be used for sensor modeling. However, the RPCs have an initial bias. Therefore, most satellite image platforms and researchers match satellite images to ground control point (GCP) information and perform RPCs bias compensation of images using the matching information. In Korea, an image-based GCP chip database built using orthographic images has been provided for the georegistration of various satellite images. In this manuscript, RPCs bias compensation of KOMPSAT-3A satellite images was performed using the GCP chips, and then, the possibility of automation of RPCs bias compensation through GCP chips was analyzed. Image matching, such as area-based and edge-based techniques, was used, and the results of RPCs bias compensation using the GCP chips were analyzed through experiments for various regions. The automated compensation in both area-based or edge-based matching performed well, achieving accuracy within 1.5 pixels for test areas with various topographic features. However, the forest area posed a challenge, requiring new GCP chips with rich feature information. In addition, edge-based matching showed potentials to overcome large seasonal differences, including snow cover.


I. INTRODUCTION
High-resolution satellite images have the potential for generating geospatial data because of their extensive ground coverage, accessibility, and repeatability. The swath width of available satellite imagery with a high spatial resolution is over approximately 10 km. The revisit time of a low-altitude earth observation satellite with high spatial resolution depends on its latitude and the acquisition angles; The associate editor coordinating the review of this manuscript and approving it for publication was Orazio Gambino . however, a revisit time of 1 to 3 days is expected for an area at latitudes of 40 to 50 • . Therefore, satellite images with high spatial resolution are widely used for mapping and environmental monitoring.
Time series analysis using geospatial information from satellite images and geospatial information systems (GISs) is required to achieve a positional accuracy of less than a few pixels. Otherwise, positional dissimilarity with other GIS data may occur. The assignment of accurate geospatial coordinates of satellite images with high spatial resolution is defined as georegistration, which is carried out through rigorous VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ physical sensor modeling using the satellite's position, attitude, and sensor information. For fast and efficient sensor modeling, most satellite images include rational polynomial coefficients (RPCs), which are coefficients of the replacement sensor model derived from physical sensor modeling as auxiliary data. Major commercial satellite image companies such as Planet, Maxar, and AirBus provide data with 10 m root mean square error (RMSE) and 5 m and 6.5 m positional accuracies of circular error at the 90 th percentile (CE90) [1], [2], [3]. Sensor modeling does not guarantee pixel-level accuracy because the result of general sensor modeling includes measurement uncertainties from onboard global navigation satellite system (GNSS) receivers, star trackers, and gyroscopes [4]. Therefore, ground control points (GCPs) are required for postprocessing to achieve high-accuracy sensor modeling. Satellite data with low spatial resolution are often georegistered using the affine model with 2D GCPs. Nevertheless, satellite imagery with high spatial resolution requires accurate 3D GCPs with elevation information, such as the digital elevation model (DEM). Therefore, 3D GCPs that are well-distributed over the entire image space are used in the RPCs bias compensation process to model uncertainty using affine, polynomial, or thin-plate spline models in the image space [5], [6]. Traditionally, the georegistration of satellite data with high spatial resolution has been carried out manually. However, many automated approaches have been proposed based on the utilization of geospatial databases as reference data, such as light detection and ranging (LiDAR) [7], topographic maps [8], orthoimages [9], and DEMs [10], [11]. LiDAR has high vertical accuracy, but the data size is large, and the available data are limited to urban areas. The shuttle radar topography mission (SRTM) [12], WorldDEM, WorldDEM Neo [13], Arctic DEM [14], and reference elevation model of Antarctica (REMA) [15] are representative reference data for the worldwide DEM. Therefore, a DEM based on stereo data is compared to a reference DEM, and similarity transformation is carried out to adjust the positional errors. However, as reference data, DEM can only be used for georegistration between stereo satellite datasets. Topographic maps also provide good reference data because they are quite easy to acquire. However, the data format and structures are quite different for satellite images. Therefore, aerial photos or satellite orthoimages with high spatial resolution have good potential as reference data because matching between satellite data and the reference orthoimage is more efficient than the use of other heterogeneous data [16].
The conventional automatic georegistration process using reference orthoimages consists of three steps. First, reference orthoimages are projected into the image space using RPCs so that image acquisition geometry is applied to the satellite image. In this step, elevation information within the reference images is required. The geometric difference between reference images and the satellite image is minimized through image projection. Second, image matching is carried out considering the error range of the initial RPCs [17], [18].
Finally, RPC bias compensation is used to remove outliers based on the false image matching results. To improve the performance of image matching, the use of quality-controlled GCP image chips as reference images is preferred. Google Maps is often used for georegistration, but its quality has not been proven because it was created for location-based services [19]. As a global reference, Airbus provides chips from stereo synthetic aperture radar (SAR) images and space reference points (SRPs) using SPOT 6/7 data for worldwide coverage [20]. The quality of an SRP that provides a dense chip distribution of at least 1 point per 2 km 2 is managed with less than 3 meters of positional accuracy (CE90) and 2 meters of ground sampling distance (GSD). TerraSAR-X GCPs with two standard accuracies, including 1 m and 3 m accuracies, are also provided [21], [22]. These are created using stereo SAR technology to collect positional information for highly reflective targets such as roads and facilities.
As local reference data, the national geographic information institute (NGII) of South Korea generated more than 23,000 chips from aerial orthophotos of all of South Korea. The match rate and achievable accuracy of georegistration between GCP chips and satellite images must be analyzed for the utilization of GCP chips from aerial orthoimage databases in automated georegistration. Therefore, we tested GCP chips from the orthoimage database to georegister KOMPSAT imagery of urban and forest areas. GCP chip projection was carried out using initial RPCs with elevation information from 5-and 10-m digital surface models (DSMs) from national topographic maps. For image matching, normalized cross-correlation (NCC) and relative edge cross-correlation (RECC) were tested to utilize the brightness and edge information [7], [23]. This manuscript is structured as follows. In Section 2, the methodology is described. The experimental results are presented for KOMPSAT images in Section 3 and the conclusion is presented in Section 4.

II. METHODOLOGY
A flowchart of the study methods is depicted in Fig. 1. Given the original image, the associated RPCs, GCP chips, and DSM, the chips were projected into the image space to align them. In this case, 3D coordinates over the chip must be transformed into the satellite image space. Then, the projected chips were matched with satellite images within the search range for the correct position of the chip in the images. Successful matching with outlier removal was used for error modeling over the entire image. In this way, the error pattern of RPCs was defined for refined RPCs.

A. GCP CHIPS IN KOREA
In the National Aerial Orthoimage database of NGII, for accessible areas, GCP chips with a 1,027 × 1,027 pixel size and a 25 cm GSD were generated, and for inaccessible areas, 1.0 m chips with a 257 × 257 pixel size were generated, as shown in Table 1. The abovementioned GCP chips were provided in the form of an image patch, including map coordinates as well as the altitude information of the center location of the GCP chip. Meanwhile, some GCP chips were not of adequate quality for georegistration, although all GCP chips were created through orthographic images with high positioning accuracy, as shown in Fig. 2. Fig. 2(a) and (b) represent chips of adequate quality with various features, such as roads and coastlines. However, some mountainous areas, as shown in Fig. 2(c) and (d), did not have spatial characteristics comparable to those of satellite images; these areas exhibited homogeneous characteristics. In addition, the texture or pattern of mountainous terrain shown in Fig. 2(c) and (d) may be sensitive to changes in seasons or time. These were not suitable for image matching and should be appropriately handled in automated processing.
Since most of South Korea consists of mountainous and vegetated areas, we determined that an additional high-quality GCP chip was required to generate accurate orthoimages in forest areas. Therefore, we generated GCP chips suitable for image matching using existing precise orthogonal images. Because GCP chips play a role in performing geometric correction and compensating for the RPC bias of satellite images, image patches composed of many feature points were required to match the image. Therefore, the GCP chip and various feature points in the GCP chip had to be automatically generated based on the region. The Harris corner point algorithm was used to extract the meaningful key points, and the result of extracting the key points in generating the GCP chips in Fig. 3(a) is shown in Fig. 3(b) [24].
However, in most mountainous areas, it is difficult to apply the image matching technique because the texture of the  topography is unclear. Therefore, using the DEM, regions with a slope of 10 degrees or more were removed from the GCP chip. In addition, only areas containing more than 10% feature points in the 1, 027 × 1, 027 pixel size image were created as the final GCP chip. Fig. 4 shows an example of one of the 102 generated GCP chips.

B. GCP CHIP PROJECTION
If georegistration is not applied, satellite imagery contains various systematic and random geometric errors, such as acquisition angle and positional errors, sensor distortion, and topographic relief errors. Because of these factors, most satellite images represent geometric differences, including scale, skew, and projection distortion, compared to orthorectified GCP chips in map coordinates. Image matching of the GCP chip with ground coordinates is essential for generating a VOLUME 10, 2022 satellite image with map coordinates using the georegistration process. However, the GCP chip was built using orthorectified images based on height information. Therefore, to perform matching with the satellite image, the GCP chip must be reprojected through the form of two-dimensional coordinates of the satellite image using the initial RPC information. We refer to this process as GCP chip projection. GCP chip projection can minimize the geometric dissimilarity that applies geometric errors to GCP chips aligned within the target satellite data. Generally, the RPC model, which is based on a nonlinear model with 78 coefficients, is defined by equation (1) and used to compute the image coordinate from the ground coordinate (φ, λ, h) [25]. The ground coordinates are normalized into (U , V , W ) to compute the normalized image coordinates (Y , X ) in equation (1). Then, (Y , X ) are used to compute image coordinates (l, s). Using equation (1), it is possible to estimate the coordinates of the image corresponding to the ground coordinates of the GCP chip. However, projecting all the pixels within the GCP chip comes with a considerable computational cost. Therefore, the remaining pixels were interpolated based on the four corner pixels of the GCP chip, which were projected using the RPCs as equation (1), to reflect the deviation of the height information in the GCP chip, and every five pixels along the sample and line directions in the GCP chip, which was associated with the digital surface model (DSM), was applied to the image projection. Finally, other pixels were interpolated, as shown in Fig. 5. In the study, the pixel interval was determined using the DSM GSD, and then each pixel coordinate in the map coordinate system was used to compute image coordinates (l, s) using equation (1). The interpolation carried out for the other pixels was based on Delaunay triangulation.    Fig. 6(b) is an example of a projected GCP chip. As shown in Fig. 6(b), the projected GCP chips were rotated and skewed depending on the image acquisition angles of the target images.

C. IMAGE MATCHING FOR RPC BIAS COMPENSATION
Algorithms for performing matching between reference and target images for image alignment or image-to-image registration can be divided into two classes: feature-based and area-based matching. In feature-based matching, scale invariant feature transform (SIFT) and speeded up robust features (SURF) are representative techniques, and they are based on the extraction of key points existing in an image [28], [29]. Feature-based matching techniques are known to have high matching accuracy. However, since a relatively large number of false matching points may occur in the matching between satellite images, they have the disadvantage of requiring a high computational cost to effectively process a large number of key points. The area-based matching method is a method of performing matching by measuring the similarity between image templates. Although the matching accuracy may be lower than that of feature-based matching, it has the advantage of performing image matching at a relatively low computational cost. Furthermore, it has been effectively applied to satellite images [7]. In the this manuscript, image matching between GCP chips and satellite images was performed using normalized cross-correlation (NCC) and relative edge cross-correlation (RECC), which are representative and conventional area-based matching techniques, and RPC bias compensation was performed using the matching results.
The NCC measures the similarity between the projected GCP chip and a satellite image patch within the search area by using cross-correlation, as shown in Equation (2). The NCC ranges from −1 to +1, and an NCC greater than 0.5 generally indicates meaningful similarity. The NCC may seem antiquated, but it is still widely used as the standard method for image matching, and it performs well for geometrically and radiometrically similar images with low computational cost.
where B is the projected GCP chip and A is a satellite image within the search region (patch) with a size of w × w.B,Ā are averages of all digital numbers (DNs) in the patches. The NCC measures the similarity of DNs between the two data. However, the DNs of the GCP chip and satellite image may differ due to the acquisition time, seasonal differences, and radiometric differences, although the DNs of all image patches were normalized. Therefore, RECC can be used to overcome seasonal differences by exploiting edge information [7]. The RECC matching robust to radiometric and illumination characteristics with the low computational cost was applied using equation (3). In RECC, the corresponding edge location of the GCP chip was found by sliding the window over the satellite image. In RECC, the normalized number of overlapping edge pixels between the projected GCP chip and the edge information in the satellite image patch was computed. It was expressed with the number of overlapping edge pixels in the numerator and the total number of edge pixels in both images in the denominator. This formula is similar to that for the NCC, but the RECC value cannot be used as the threshold. Instead, the average distance between four locations in the image with the top RECC values as CV_4 in equation (4) was used as the threshold to determine if the match is sufficient. Small values, such as 2 to 3 pixels, indicated successful match results.
where B e is a window in the edge image of the projected GCP chip, A e is an edge image of the satellite image patch, and B e ij and A e ij are the digital numbers at row i and column j. CV 4 is the concentration value based on the maximum to fourth largest RECC value, and (r max ) and (r i , c i ) are the image coordinates of the positions of the maximum RECC and i th largest RECC value, respectively. Instead of directly using the RECC value, image matching was performed using the CV 4 value.

D. BIAS COMPENSATION FOR OUTLIER REMOVAL
Using the coordinate differences through the image matching results, geometric error information over the entire image can be described as in equation (6), which is the RPC bias compensation model [5]. (l, s) was determined using the initial RPC model in equation (1), and l ,s was computed by using the location difference from the image matching result between the projected GCP chip and satellite image as bias compensated image coordinates. Model parameterization depends on the satellites, but many satellites require affine transformation. Therefore, if there is a matching result for at least four GCP chips and satellite images, the coefficient of affine transformation for determining RPC bias compensation can be defined using the least square method. equation (6) can be expressed in matrix form for an image point, and the number of rows should be 2×n for n observed points.
where a 0 , a 1 , . . . ,b 2 are the coefficients of affine transformation, a 0 , a 1 , a 2 , b 0 , b 1 , b 2 are the model shift, drift, and scale to angular affinity, and a 0 , b 0 are only for the shift. Additionally, false matching that occurs during the image matching process causes an error in RPC bias compensation. Therefore, a method for removing outliers corresponding to false matching should be implemented. Outlier removal can be performed using various algorithms, such as the RANSAC and data snooping algorithms [25], [26]. In this study, we used the data snooping algorithm because the number of observation equations is less than twelve. To apply data snooping, test statistics for outlier detection were computed as shown in equation (7) for n points with an affine-based bias model (m=6). If the statistical value T j was larger than the threshold, we considered this point to be an outlier.
whereẽ is the residual computed from equation (6), P is the weight matrix of an observation, r j is the redundancy number, andẽ j and p j are the j th elements.

A. EXPERIMENTAL DATASET
KOMPSAT-3A test images were used to test the RPC bias compensation, as shown in Fig. 7. The study area is South and North Korea, and the area of each scene was approximately 184 km 2 . The test images of the four areas were divided into complex and mountainous areas.   Since the complex area in Case 1 contains various features, such as roads, cultivated areas, and buildings, Case 1 was selected as a criterion for the image matching experiment. Case 2 was selected to analyze the image matching results when additional GCP chips including various features were used. Finally, through an experiment using Case 3, we attempted to analyze the effect of the matching technique between NCC and RECC. Table 2 gives the specifications of the dataset for the four regions used in the experiment. There were 20 to 257 GCP chips in each target image, as shown in Table 2. The locations of GCP chips included in each experimental image were distributed over the entire region, and the number of GCP chips varied for each image.

B. GCP CHIP AND SATELLITE IMAGE MATCHING
First, we applied image matching to the projected GCP chips using NCC. As shown in Fig. 6(b), the projected GCP chips included null values due to the image transformation used for GCP chip projection; we clipped the center part of the chip to obtain 120 × 120 pixels. The search region within the target satellite data was set to 100 pixels, considering the positional accuracy of the KOMPSAT-3A data. In image matching, GCP chips with a correlation value of less than 0.3 were considered mismatched because the larger matching window sizes that we used in the study decreased the similarity and correlation values such that a typical value, such as 0.5 or 0.7, was not optimal in these cases. The remaining mismatched GCP chips were removed through outlier removal after matching with a low NCC value. The matching process of RECC was the same as that of NCC, but a threshold of CV4 was set to three (pixels) through trial and error. Outliers were removed with 95% statistical confidence using the affine model [5]. Note that even though the match was successful and correct, the chip could be classified as an outlier in the statistical testing of the outlier removal process. Fig. 8 shows some examples of the image matching results using NCC. Successful matching results were obtained for various geographical features, such as roads, paddies and linear features; however, matching errors resulted based on snow cover and seasonal differences in forest areas. Fig. 9 shows a sample in which RECC overcomes the land cover difference due to the snow cover, and Fig. 10 represents an example of GCP chip distribution for matched and mismatched cases about Fig. 7(b).

C. EXPERIMENTAL RESULTS AND ANALYSIS BY CASE
We carried out RPC bias compensation modeling using the matching results. Tables 3 to 8 show the residual values in the column and row directions, success rate, and the number of chips used for the modeling for each case. In these experiments, we classified a point that shows positional accuracy within 5 pixels as a successful match, and then, the matching success rates were calculated by using the ratio between the number of overall GCP chips and the number of correctly matched GCP chips. Finally, the number of GCP chips used for RPC bias compensation after outlier removal based on the correctly matched GCP chips is also described in each table. Note that some mismatched GCPs may have been included after outlier removal. Table 3 shows the results of GCP chip-based RPC bias compensation for the eighteen satellite images in Case 1. The satellite images in Case 1 include urban, agricultural, and mountainous areas. As a result of performing RPC bias compensation through image matching, more than ten GCP chips were matched in all images, and the proportion of correctly matched GCP chips was 34-67%, except for in the 18 th image. In addition, it was confirmed that the positional accuracy was approximately 1.3 (pixels) in the average column and row directions. In the 18 th image, the number of correctly matched GCP chips was relatively small, but the RPC bias  was effectively removed. Therefore, we confirmed that the NCC technique, a typical conventional method used in image matching, can be effectively performed for regions where apparent features exist.

1) RPC BIAS COMPENSATION RESULTS FOR CASE 1
For comparison, we carried out RECC for Case 1, and the results are presented in Table 4. The overall matching rate increased, but the positional accuracy was similar to that of NCC.

2) RPC BIAS COMPENSATION RESULTS FOR CASE 2
The five images in Case 2 consisted of mountainous areas. As seen in Table 5, the positional accuracy for most images was over 30 pixels. Therefore, RPC bias compensation for all images included in Case 2 was not performed correctly. Specifically, compared with the results in Case 1, the success  matching rate in Case 2 was also very low. Since mountainous areas consist of homogeneous features, no GCP chip was accurately matched using the NCC technique.
We applied RECC instead of NCC, but it also failed for Case 2.   To minimize large mismatches when NCC was applied to homogeneous areas, additional GCP chips were created, as mentioned in Section 2 of this paper. As seen in Table 6, the positional error in the column and row directions was reduced for all five sets of experimental data. In particular, the success matching rate was also significantly improved to 20 to 40%. Therefore, by performing image matching using the GCP chip, we confirmed that the performance of the NCC can be improved by using the new GCP chip, including its various features. However, compared to the complex area, the results of RPC bias compensation in the homogeneous area were of relatively low quality.

3) RPC BIAS COMPENSATION RESULTS FOR CASE 3
Case 3 consisted of a mountainous areas similar to those in Case 2. However, compared to the mountainous region in Case 2, linear features such as rivers were more diverse in Case 3. Table 7 shows the result of performing RPC bias compensation using NCC. As seen from this table, some images exhibited positional accuracy of approximately 1 pixel, while the others showed very poor results. In the case of these data, although a sufficient number of GCP chips were matched and used for modeling, low positioning accuracy was achieved, meaning that the NCC technique mismatched most of the GCP chips included in the test data. Table 8 shows the results of applying the RECC method as an alternative to the same experimental data.
Image matching was effectively achieved when the RECC technique was applied to the image with poor NCC matching.
Ultimately, experiments confirmed that the GCP chip and satellite images were effectively matched using properly applying area-based matching methods such as NCC and RECC. In addition, it was practical to add a GCP chip including meaningful points to perform image matching, especially over a homogeneous area where it was challenging to extract meaningful fey key points, such as a forested area.

IV. CONCLUSION
In this study, we tested orthoimage-based GCP chips for RPC bias compensation of high-resolution satellite images such as those from KOMPSAT to meet the requirement of less than a few pixels of positional error. The automated approaches consisted of GCP chip projection, image matching in the target image space, and bias compensation with outlier removal. For image matching, NCC, area-based matching, was used as the primary method, and RECC, edge feature matching, was also used as an alternative. We drew the following conclusions from the experiments performed on many KOMPSAT datasets: 1) Automated RPC bias compensation using GCP chips worked well for target images, including urban areas with one-pixel level residuals.
2) Area-based matching such as NCC often failed over target images with a homogeneous area. Mismatches occurred in areas with changes in land cover, snow cover, local fog, and shadows. Seasonal land cover changes may be overcome using edge feature matching such as RECC or through the generation of additional GCP chips.
3) In the GCP chip generation step, it is essential to consider automated image matching performance and use, for example, chips with many linear features and multiple chips with seasonal changes.