Fractal Theory Based Stratified Sampling for Quality Assessment of Remote-Sensing-Derived Geospatial Data

The stratified sampling is widely used in quality assessment of remote-sensing-derived geospatial data (RSGD). Because of the different stratification indicators (also called the stratification variables) used in stratified sampling, it will lead to different evaluation results. By using fractal theory, this article proposes a stratified sampling method based on fractal (SSF) for quality assessment of RSGD. As a stratification variable, fractal dimension is related to and independent of the study variable in the quality assessment of RSGD. This method can quantitatively and accurately stratify the population, which leads to minimizing the intra-stratum variance, acquiring higher estimation accuracy, and estimation efficiency. The proposed SSF method in this article is transformed into three formulated problems: the quantitative calculation of fractal, the optimal solution of the stratum boundary value, and the configuration of sample sizes. The experiment shows a quantitative performance comparison of SSF, stratified sampling based on class, and sample random sampling using the South Sudan Global Core Vector Database 2020. Design effect and root-mean-square error provide a quantitative assessment of the performance in this study. The experimental results verify the feasibility and applicability of the SSF proposed in this article. It also shows higher estimation accuracy and more economical cost.

SECTION I. Introduction

With the rapid development of remote sensing technology in the world, the amount of data generated by Terra, Landsat, Spot, NOAA, HJ-1B, GF, and other remote sensing satellites has increased exponentially, and the types of products derived from these data have also increased more and more. The remote-sensing-derived geospatial data (RSGD) are produced based on remote sensing technology and other technologies, including digital maps, digital elevation models, landcover products, land surface temperature products, etc. [1], for example, the VCF data developed by the US Geological Survey (USGS), the University of Maryland and other institutions [2], the MODIS-MCD45 burned area product [3], the USGS National Land Cover Database [4], OpenStreetMap (OSM), etc. [5]. The quality of RSGD, which is widely used in various industries, is getting more and more attention. The quality of RSGD is a key factor that affects the end use. For example, in rice planting acreage estimation [6], deforestation acreage statistics [7], global land cover classification statistics [8], and even in autonomous driving, the quality of RSGD will directly affect the accuracy, reliability, inference, and decision-making at the time of using it [9]. Therefore, the quality assessment of RSGD can provide users with evidence to confirm the quality level and determine whether the quality can meet the expected requirements [10]. In the era of Big Data, the time, money, manpower, and equipment spent on all inspection items of each one for quality evaluation could not be borne by both producers and users. In quality evaluation, sampling inspection is an economical and efficient method, which selects a part of the whole for quality inspection, and then infers the overall data or products.

In the research of sampling methods, the commonly used methods are simple random sampling (SRS), systematic sampling, cluster sampling, and stratified sampling [11]. Stratified sampling is a method that divides the population into multiple nonoverlapping subpopulations (each subpopulation is called a stratum) according to stratification indicators (also known as stratification variables), and sampling independently from each stratum [12], [13]. Because it can obtain higher estimated accuracy than SRS, stratified sampling is adopted in large amounts in the work of land cover statistics and accuracy assessment [14], [15], [16].

When the population is stratified by different stratification indicators, different estimators will be obtained when the sample size is the same. The principle of stratified sampling should make the intrastratum variance as small as possible. Xie et al. [1] designed a multilevel stratified spatial sampling approach for the quality assessment of RSGD; Giree et al. [17] used the reference data as the stratification indicator to estimate the gross forest cover loss; Olofsson et al. [18] used the Köppen climate classification and population density as the stratification indicator to validate the global land coverage classification data. These studies follow the principle of large interstratum variance and small intrastratum variance. When these methods comply with the principle, the research works show that the higher the correlation between the stratification variables and the target variables (study variable) is, the higher the final estimation accuracy is [19], [20]. The stratification indicator adopted by Xie et al. [1] depends on the product to be evaluated, but the quality of the product to be evaluated is unknown, which leads to an unknown impact on the estimation accuracy. The stratification indicator adopted by Giree et al. relies on reference data, which would fail when the reference data cannot be obtained. The stratification indicators adopted by Olofsson et al. [18] are climate types, whose benefit improvement of final verification is limited because the correlation between climate types and estimators is unknown. Therefore, exploring a strict stratification indicator is an important research content of stratified sampling. In the quality assessment of RSGD, ideally, stratifying directly according to the quality assessment value of each element is the optimal stratified sampling design [20], [22]. However, in the sampling stage, the quality evaluation value of each element is unknown. If that is known, sampling is not needed to estimate the population quality [23]. Therefore, it is an ideal situation that the target variables are set as stratification variables [24]. The stratified sampling based on class (SSC) with the classification of elements as the stratification indicator is the most widely used sampling method at present. However, the problem with this method is that the correlation between element classifications and quality is not obvious, which has an impact on the reliability of the final estimator. In this case, it is necessary to study and explore a stratification indicator suitable for RSGD quality evaluation. This stratification indicator is not limited and dependent on the quantity to be evaluated, which has a strong correlation with the indicator.

Fractal is a theory across natural science and social science, which was founded in the 1970s. It was first proposed by Mandelbrot [25] in a paper published in the Science in 1967. It can describe some irregular, unstable, or complex objects in nature. Fractal is a common phenomenon whose theory is a branch of nonlinear scientific theory. It is mainly used to find the laws hidden behind complex and chaotic phenomena. Different from strict mathematical fractal, natural objects usually do not show accurate self-similarity [26]. Based on the above characteristics, fractal geometry provides great potential in the representation, measurement, and analysis of complex remote sensing images and geographical elements [27], [28], [29]. For example, Jiang and Brandt discussed the application of fractal in geospatial topology and scale analysis [30]. Bretar et al. [31] tried to use fractal dimension to characterize the surface roughness of volcanic terrain remote sensing images. Cheng et al. [32] studied and calculated the landscape self-similarity index of Taiwan Island based on fractal and obtained the conclusion that the landscape self-similarity index of Taiwan Island changes with altitude. Cosh and Brutsaert [33] calculated the fractal dimension of vegetation density in different land classifications through wavelet analysis, spectral analysis, and other methods. Read and Lam found that fractal dimension is useful to characterize the spatial complexity of Landsat data [34].

In prior studies on fractal-based sampling methods, Ferrandino [35] conducted a study on measuring spatial aggregation in binary epidemics, utilizing a nested fractal design where the sample point locations were determined by a fractal pattern based on squares. The study compared and analyzed the advantages of this nested fractal design in contrast to the grid sampling and the cluster sampling [35]. Marsh and Ewers designed a scalable sampling design utilizing fractal geometry. They employed a sampling frame consisting of a fractal pattern based on equilateral triangles, with the objective of ecological surveys quantifying β-diversity [36]; Jiang et al. [37] proposed a novel sampling scheme based on Cantor sets, a classic fractal. This scheme utilizes a well known and precise fractal structure to accelerate parallel simulation of multithreaded applications [37]. Previous studies on fractal-based sampling methods used the principle of self-similarity to determine sampling locations or sampling frames by means of triangular, square, or hexagonal grids with changeable side lengths. Nonetheless, these methods can only be applied to point elements and are not suitable for assessing the quality of linear or polygonal elements in RSGD.

In the production of RSGD, the higher the fractal dimensions are, the more disordered and complex the corresponding elements are [30]. The more disordered and complex the corresponding elements are in RSGD, the more difficult it is to produce such elements that meet the quality requirements (the degree of difficulty is related to this). The quality assessment of RSGD is based on whether each element meets the quality requirements. It gives the feasibility of using fractal dimension as stratification indicator [38].

This article aims to integrate fractal theory and propose a stratified sampling method based on fractal dimension. This method can use fractal dimension to quantitatively describe the structure of the data to be evaluated. The purpose of this experiment is

1. to verify the feasibility of fractal dimension as a stratification indicator by comparing stratified sampling method based on fractal (SSF) with SRS;

2. to study whether the proposed method can provide more effective quality assessment through the comparison with different sampling methods and different stratification indicators;

3. to explore whether it can improve the sampling efficiency through the comparison with the stratification indicator.

The rest of this paper is organized as follows: in Section II, the SSF method for RSGD quality assessment is studied, and the theoretical proof and conceptual model of the method are given. After that, the experimental area and experimental data are introduced; in Section III, the proposed SSF method is quantitatively compared with SRS and SSC; in Section IV, the experimental results of the previous section are analyzed and discussed to verify the proposed SSF in this article; in the last section, the performance of SSF method is discussed and summarized.

SECTION II. Methodology

A. Theoretical Basis of the SSF Method

For the quality assessment of RSGD using traditional stratified sampling, the set of each element (also called individual) in the RSGD is defined as a population M of size N, and ${q}_i$(i = 1, 2, 3…, N) represents the quality assessment value of each element. Next, the population is divided into h layers according to the value of ${q}_i$, and samples of size ${n}_h$ are selected from each layer to form a sample population m of size n. The unbiased estimator for the population quality assessment value is constructed by the quality assessment value of the samples, and the unbiased estimator for the population variance of the quality assessment value is given by \begin{equation*} \hat{V}\left({{{\bar{q}}}_{st}} \right) = \mathop \sum \limits_{h = 1}^L W_h^2\left({1 - \frac{{{n}_h}}{{{N}_h}}} \right)\frac{{s_h^2\left(q \right)}}{{{n}_h}} \tag{1} \end{equation*} where N is the size of population, ${N}_h$ is the size of the stratum h (h = 1 to L), ${n}_h$ is the sample size of the stratum h, $W_h^{}$ is the weight of the stratum h, $s_h^2(q)$ is the sample variance of the quality assessment value in the stratum h [39]. The optimal stratified sampling method should minimize the variance estimator $\hat{V}({{{\bar{q}}}_{st}})$ for the study variable ${q}_i$ [40].

It can be observed that when the stratum weight $W_h^{}$ is fixed, $s_h^2(q)$ shows a positive correlation with $\hat{V}({{{\bar{q}}}_{st}})$. Therefore, reducing $s_h^2(q)$ as much as possible can minimize $\hat{V}({{{\bar{q}}}_{st}})$. From the definition of variance, it can be known that the sample variance of the quality evaluation value in the stratum h is \begin{equation*} s_h^2\left(q \right) = \left({\frac{{\mathop \sum \nolimits_{i = 1}^{{n}_h} {{\left({{q}_i - {{\bar{q}}}_h} \right)}}^2}}{{{n}_h - 1}}} \right) \tag{2} \end{equation*} where the study variable ${q}_i$ is used as the stratification indicator (or stratification variable). However, in practical application, the ${q}_i$ to be studied or investigated is an unknown quantity.

To address this issue, this article proposes the SSF method, which uses the fractal dimension as a stratification indicator or auxiliary variable, as it is easily obtained. The fractal dimension value of each element in the population M is denoted as ${f}_i$(i = 1, 2, 3…, n), then the sample variance of the fractal dimension value in the stratum h is \begin{equation*} \mathrm{s}_{h}^2\left({{f}_i} \right) = \left({\frac{{\mathop \sum \nolimits_{i = 1}^{{n}_h} {{\left({{f}_i - {{\bar{f}}}_h} \right)}}^2}}{{{n}_h - 1}}} \right). \tag{3} \end{equation*}

If there is a positive or negative correlation between ${f}_i$ and ${q}_i$, minimizing $s_h^2({{f}_i})$ or maximizing $s_h^2({{f}_i})$ can make $s_h^2({{q}_i})$ as small as possible. Considering that ${f}_i$ and ${q}_i$ correlation coefficients is \begin{equation*} {p}_{fq} = \frac{{Cov\left({f,q} \right)}}{{\sqrt {D\left(f \right)} \sqrt {D\left(q \right)} }} = \frac{{Cov\left({f,q} \right)}}{{{s}_h\left(f \right){s}_h\left(q \right)}} \tag{4} \end{equation*} we can show that \begin{equation*} {s}_h\left(q \right) = \frac{{Cov\left({f,q} \right)}}{{{s}_h\left(f \right){p}_{fq}}}. \tag{5} \end{equation*}

It can be seen from (5) that the ${s}_h(q)$ has a negative correlation with the correlation coefficient ${p}_{fq}$. If there is a correlation between the auxiliary variable ${f}_i$ and the study variable ${q}_i$, $\hat{V}({{{\bar{q}}}_{st}})$ can be reduced and the sampling accuracy can be improved. Moreover, the higher the correlation between ${f}_i$ and ${q}_i$ is, the higher the improvement of sampling accuracy is. Therefore, according to (1)–​(5), the proof of the feasibility of the SSF method proposed in this article can be transformed into a proof of the correlation between the fractal dimension value ${f}_i$ and the quality assessment value ${q}_i$ of the same element.

Fractal dimension is a quantitative description of fractals. Different from the traditional Euclidean Space, it can quantitatively describe geospatial objects in a continuous noninteger dimension. The size of its value is closely related to “complexity,” “roughness,” etc., [41], [42], [43], [44]. It is difficult to directly prove the correlation between the fractal dimension and the quality assessment, so this article introduces the Shannon Entropy to prove it.

The Shannon Entropy is a rigorous measure of uncertainty in discrete random variables [45], [46], [47], [48]. When the quality of RSGD is evaluated by the SSF method, the quality assessment value of each element can be defined as a discrete random variable. Thus, the Shannon Entropy ${U}_i$ of each element quality assessment is the quantitative characterization of the quality uncertainty. According to the physical meaning of Shannon Entropy, the larger its value ${U}_i$ is, the higher the uncertainty is and the lower the quality assessment ${q}_i$ is. The negative correlation between the Shannon Entropy ${U}_i$ and the quality assessment value ${q}_i$ is evident. If it can be proved that the fractal dimension value ${f}_i$ of the same element has a correlation or an equivalence relation with the Shannon Entropy ${U}_i$, there is a correlation between ${f}_i$ and ${q}_i$, that means the proof for the feasibility of the SSF method proposed in this article is completed.

It is known that each element in the population M is expressed as ${M}_i$ (i = 1, 2, 3…, N), and the corresponding fractal dimension can be expressed as \begin{equation*} {f}_i = \mathop {\lim }\limits_{\varepsilon \to 0} \frac{{\ln {K}_i}}{{ - \ln \varepsilon }} \tag{6} \end{equation*} where $\varepsilon $ represents the measure in a region of size $\varepsilon $, and ${K}_i$ represents the number of times that the ith element needs to be measured under the $\varepsilon $ measure. The Shannon Entropy [49] corresponding to ${M}_i$ can be expressed as \begin{equation*} {U}_i = - \mathop \sum \limits_{i\ = \ 1}^K {P}_i\ln {P}_i. \tag{7} \end{equation*}

Considering the uniform distribution of disjoint $\varepsilon $ under the same measure $\varepsilon $, the Shannon Entropy can be written as \begin{equation*} {U}_i = - \mathop \sum \limits_{i\ = \ 1}^K \frac{1}{{{K}_i}}\ln \frac{1}{{{K}_i}} = \ln {K}_i. \tag{8} \end{equation*}

According to (6) and (8), we can establish a simple relationship between ${f}_i$ and ${U}_i$ that can be written as \begin{equation*} {f}_i = \mathop {\lim }\limits_{\varepsilon \to 0} \frac{{{U}_i}}{{ - \ln \varepsilon }}. \tag{9} \end{equation*}

Thus, the fractal dimension ${f}_i$ grows as the Shannon Entropy ${U}_i$ from (9), which means both are positive correlation.

Through (6)–​(9), the correlation between fractal dimension value ${f}_i$ and the Shannon Entropy value ${U}_i$ of the same element in RSGD is proved, which means the feasibility of the SSF method proposed in this article is proved.

B. Implementation of the SSF Method

The conceptual flow chart of the SSF sampling method for RSGD quality evaluation is in Fig. 1. The implementation of SSF method is mainly divided into four steps.

1. First, defining the population and samples, that is, each geospatial object of the data to be evaluated corresponding to each element is defined as population M of size N, and the elements of size n (n≤N) selected from M are called the sample population m.

2. Calculating the fractal dimension of each element in the population and record them as ${f}_i$(i = 1, 2, 3…, N).

3. According to the fractal dimension value ${f}_i$ of each element, it is divided into nonoverlapping parts called strata with a size of h. Each element corresponding to each ${f}_i$ is also divided into corresponding strata.

4. The sample size ${n}_h$ of each stratum is calculated according to a certain sample allocation rule. And then the samples are distributed in each stratum according to the simple random principle.

Fig. 1.

SSF conceptual flow chart.

C. Calculation of the Fractal Dimension

There have been many research works on calculating the fractal dimension, including Variogram method [50], [51], Divider method [52], [53], Box-counting method [54], [55], the Power Spectral method, etc., [56], [57], [58]. The Box-counting method uses square grids with different side lengths ${\rm{\varepsilon }}$ to cover the studied object. When the ${\rm{\varepsilon }}$ of the square grid changes, the number $N({\rm{\varepsilon }})$ of grids covering the studied object changes accordingly and can be written as \begin{equation*} K\left({\rm{\varepsilon }} \right) \sim {{\rm{\varepsilon }}}^{ - f}. \tag{10} \end{equation*}

In a double logarithm graph, a series of points are linearly fitted. The absolute value of the slope of this line is the fractal dimension $f$. The $f$ can be calculated as \begin{equation*} \lg K\left({\rm{\varepsilon }} \right) = - f\lg {\rm{\varepsilon }} + C \tag{11} \end{equation*} where ${\rm{\varepsilon }}$ is the grid side length, $K({\rm{\varepsilon }})$ is the grid number, and $C$ is the undetermined constant. RSGD is divided into vector data and raster data. Raster data itself is a matrix composed of squares. Vector data can be processed by constructing a square grid. Therefore, in SSF method, the Box-counting method is used to calculate fractal dimension. Figs. 2 and 3 show the calculation results of the traffic and water linear products of South Sudan Global Core Vector Data (GCVD) 2020, which is a kind of RSGD, by using the Box-counting method. The proportion of elements with high fractal dimension in map X is significantly higher than that in map Y.

Fig. 2.

Map X and its corresponding fractal dimension histogram.

Fig. 3.

Map Y and its corresponding fractal dimension histogram.

D. Determining the Number and Boundary of the Strata

To evaluate the quality of RSGD with SSF, the population needs to be stratified. In the previous subsubsection, the most appropriate stratification indicator-fractal dimension has been studied and determined. The following problem is: how many strata should be divided and the boundary between strata should be determined. William G. Cochran deduced the relationship between the population variance of unbiased estimator and the stratum number in 1972. Besides, it is proved that when L is large and 6, the reduction rate of V will be greatly slowed down. Fuller [59] and Zseby [22] also showed that the larger L is, the higher the gain of stratified sampling is,. When L is greater than 4, the gain of stratified sampling tends to be stable.

After the number of strata is determined, it is necessary to solve the problem of strata boundary determination, that is, the value of stratification indicator x, which is the basis for determining strata boundary.

How to determine the stratum boundary? Dividing the cumulative square root of the distribution density function f(x) of the stratification variable x equally can make the variance of x in each stratum as small as possible. This is in line with the principle of small intrastratum variance and large interstratum variance. When the population is reasonably stratified, the sample size allocation can be carried out based on the number of subpopulations in each stratum.

E. Sample Size Allocation

After the boundary of each stratum is determined, the samples need to be reasonably distributed to each stratum in a certain way. In stratified sampling, the different methods of sample allocation in each stratum will have a certain impact on the accuracy of estimators. The main methods are equal allocation, proportional allocation, and Neyman allocation [24]. Equal allocation makes the sample size ${n}_h$ equal in each stratum, without considering the population characteristics. Proportional allocation is to make the sample size proportional to the subpopulation size ${N}_h$ of each stratum. The sample size of each stratum ${n}_h$ can be written as \begin{equation*} {n}_h = n \times {w}_h = n \times {W}_h = n \times \frac{{{N}_h}}{N}. \tag{12} \end{equation*}

Neyman allocation assumes that the sampling cost is equal. When the sample size n is fixed, the sample size of each stratum ${n}_h$ can be written as \begin{equation*} {n}_h = n \times {w}_h = n \times \frac{{{W}_h{S}_h}}{{\sum {W}_h{S}_h}}. \tag{13} \end{equation*}

F. Effectiveness Evaluation Method of the SSF Method

To assess the advantages and disadvantages of the SSF method, it is necessary to evaluate the gain it generates. Different sampling methods have different effectiveness of estimators. The advantages and disadvantages of different methods are evaluated by design effect (DEFF) and root-mean-square error (RMSE).

DEFF is an index to measure the effect of sample method proposed by Kish [60], which is called design effect. DEFF can be calculate by \begin{equation*} \text{Deff} = \frac{S}{{{S}_r}} \tag{14} \end{equation*} where S is the estimator variance of the sample method to be evaluated, and ${S}_r$ is the estimator variance of SRS under the same sample size.

The idea of using RMSE for evaluation is when the sample size n is the same, the observed value of the sample quality assessment obtained by method A is $\{ {{A}_1,{A}_2, \ldots \ldots,{A}_n} \}$, the observed value of the sample quality assessment obtained by method B is $\{ {{B}_1,{B}_2, \ldots \ldots,{B}_n} \}$, and the true value of the overall quality assessment is $\{ {{T}_1,{T}_2, \ldots \ldots,{T}_n} \}$ \begin{align*} \text{RMSE}\left(A \right) &= \sqrt {\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n {{\left({{A}_i - {T}_i} \right)}}^2} \tag{15}\\ \text{RMSE}\left(B \right) &= \sqrt {\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n {{\left({{B}_i - {T}_i} \right)}}^2} . \tag{16} \end{align*}

If RMSE(A) < RMSE(B), the estimator obtained by sample method A is closer to the true value, with less dispersion degree and higher accuracy.

SECTION III. Empirical Case Studies

A. Study Area

South Sudan selected for the experimental study area is in northeast Africa, between longitude 24°9'7“–35°52'12” and latitude 3°28'52“–12°13'8.” It belongs to zone 35 and 36 of UTM projection 6°, and the central meridian is longitude 27° and 33°, respectively. It borders Ethiopia in the East, Kenya, Uganda, and the Democratic Republic of the Congo in the south, the Central African Republic in the West, and Sudan in the north. South Sudan has a tropical grassland climate, and more than 95% of the whole territory is humid and semi-humid areas, with high humidity and a lot of rainfall. The rainy season is from May to October every year, and the temperature is 20–40 °C; the dry season is from November to April, and the temperature is 30–50 °C. Besides, forest resources of South Sudan are very rich.

The study data is the traffic and water linear products of South Sudan GCVD 2020, which is the vector data expressing geographic information elements in the form of lines. The products select main elements according to demand, importance, and operability. Six maps with the same size are randomly selected from the study data as the experimental data, which is consistent with the assessment unit of the quality evaluation organization. The size of each map obtained by measuring tools is 27500 m × 18400 m. The study area and experimental data of this experiment are shown in Fig. 4.

Fig. 4.

Study area and experimental data.

B. Experiment Design

In this experiment, six maps A, B, C, D, E, and F were compared by three methods: SSF, SSC, and SRS. The experimental maps were sampled respectively, and the highest sampling ratio of each map was designed as W% of the population. Each method was used to sample according to the sample sizes of W%. The quality was evaluated according to the reference data, and the corresponding correct rate of different sampling methods under the same sampling ratio was calculated. We repeated the experiment 50 times as the observation values, and the reference data were the reference values. According to (15) and (16), the RMSE of the corresponding correct rate under the three sampling methods can be calculated, and finally, the benefits of the three sampling methods can be evaluated. It should be noted that

1. to make the comparison between SSF and SSC meaningful in this experiment, the number of strata of the two methods in each map should be kept equal;

2. after sampling, the quality inspection items of the samples are checked, and the variable ${Y}_i$(i = 1, 2,……,n) is used to represent the result of inspection item. This variable can be taken as 0 and 1. “Y = 1” indicates that the inspection item is correct, “Y = 0” indicates that the inspection item is defective. In order to facilitate the experimental calculation, the quality assessment value of the study variable is simplified to the correct rate. The correct rate ${p}_r$ of the samples can be calculated by \begin{equation*}{p}_r = \frac{r}{n} = \frac{1}{n}\mathop \sum \limits_{i\ = \ 1}^n {Y}_i \tag{17} \end{equation*}

where $r$ is the number of correct samples. According to the statistical theory, the sample correct rate ${p}_r$ is an unbiased estimator ${P}_r$ of the population

3. to comprehensively evaluate the performance of this experiment under different sample sizes, the W% values were designed as 2%, 4%, 6%, 8%, 10%, 12%, 14%, 16%, 18%, and 20%, respectively. The upper limit of sample size is set to 20% of the population is that, in RSGD quality evaluation, it is very high cost to reach 20%. Usually, the sample size will not exceed 20%; on the other hand, repeating the experiment 50 times can reduce the sampling error, make the experimental results closer to the reference data, and avoid insufficient reliability of the conclusion due to too few observed values [61].

C. Experimental Data

The population and samples: The population sizes of the six maps are 1778, 2514, 2811, 1493, 1197, and 1299, respectively, as shown in Table I. To ensure a fair comparison, it is necessary for the sample size to be consistent across the three sampling methods. The specific sample size of each map is shown in Table I.

TABLE I Population and Sample Size

The reference data: Referring to Google data and OSM data, we evaluated experimental data quality by means of expert judging all elements of each map. Each element was judged by three experts independently, and two or more consistent results were deemed to be correct, otherwise, it was wrong. According to Section III-B and (17), we calculated the population reference correct rates (or “true” correct rates) of each map (see Table II).

TABLE II Experimental Data Reference Correct Rate

D. Stratification Result Based on Fractal

With the help of ArcGIS software, the experimental data were processed, and the fractal dimension of each element in the six maps was calculated according to (6). The classifications of Map A, B, C, D, E, and F was 6, 5, 5, 5, 6, and 6. Therefore, when the SSF method was adopted, the number of strata of the six maps also took the same value. By the cumulative square root of frequency method, we calculated the cumulative square root values of the six maps, which were 244.4316, 283.7608, 238.3648, 303.4573, 203.7184, and 210.4429, respectively. They were stratified according to the number of strata above. The stratification results of the two methods are shown in Fig. 5.

Fig. 5.

Stratification results of the experimental data. (A), (B), (C), (D), (E), and (F) are the results of Map A, B, C, D, E and F, respectively, by the SSF method. (a), (b), (c), (d), (e) and (f) are the results of Map A, B, C, D, E and F, respectively, by the SSC method.

E. Sampling and Assessment

After obtaining the stratification results, according to (12), the proportional allocation was used to calculate the sample size of each stratum under the ten categories of sample size designed in Section III–B (2%, 4%, 6%, 8%, 10%, 12%, 14%, 16%, 18%, and 20%) respectively. In the SSF, the fractal dimension was used as the stratification indicator. The population of each map was divided according to Table I, and then samples were taken at each stratum through the random principle.

This experiment was a comparative analysis of the gain of different methods, which needed to adopt the same quality evaluation principle. Through the quality evaluation model and reference data, the quality inspection items of each sample in maps A, B, C, D, E, and F were evaluated [62]. Equation (17) can calculate the correct rate of each sampling ratio. We repeated the above process 50 times to obtain a group of observed values of the correct rate and then compared the reference data to calculate the DEFF and RMSE of the correct rate (see Table III).

TABLE III DEFF and RMSE Results of the SSF, SSC, and SRS Method

SECTION IV. Discussion

Fig. 6 shows the average DEFF values of the SSF method, the SSC method, and the SRS method in each experimental map. The x-axis represents the map name, the y-axis represents the DEFF value. The black solid line, blue solid-dotted line, and the red dotted line represent the ranges of DEFF values under the SSF method, the SSC method, and the SRS method, respectively.

1. It can be seen from Fig. 6 that the SSF method's DEFF result is less than 1, indicating the effectiveness of the SSF method.

2. All the SSF method's average DEFF values are less than the SSC method's and the SRS method's, indicating the advantages of the SSF method.

3. The DEFF range of the SSF method is smaller than that of the SSC method in each map, indicating that the SSF method is less influenced by the sample size.

4. The average DEFF fluctuation of the SSF method is less than that of the SSC method, indicating that the SSF method is less affected by the spatial location.

Fig. 6.

DEFF value ranges and the average DEFF value of the SSF method, the SSC method, and the SRS method in map A, B, C, D, E, and F.

Fig. 7 shows the experimental RMSE results of the SSF method, the SSC method, and the SRS method in the six experimental maps. The x-axis represents the sampling ratio, the y-axis represents the RMSE, connecting the RMSE values of the same method with different sample ratios. The black solid line represents the SSF method, the blue solid-dotted line represents the SSC method, and the red dotted line represents the SRS method.

1. By comparing the black solid line and the red dotted line, it is evident that the RMSE of the SSF method is consistently lower than that of the SRS method across all sampling ratios. It indicates that the SSF method proposed in this article is significantly better than the SRS method.

2. Through the comparison between the black solid line and the blue solid-dotted line, it is observed that the RMSE of the SSC method is slightly lower than that of the SSF method, except for the 10% sample rate in Fig. 7(a), the 10% sample rate in Fig. 7(c), the 4% sample rate in Fig. 7(d), and the 16% sample rate in Fig. 7(e). Most other experimental results show that the RMSE of the SSF method is lower than that of the SSC method. It shows that the SSF method proposed in this article is better than the commonly used SSC method.

3. It can be seen from the three lines that with the increase of sampling ratio, the RMSE of the three methods decreases and gradually converges to 0, among which the SSF method converges the fastest.

4. When the RMSE value is same, the sampling ratio corresponding to the SSF method is the smallest, indicating that the SSF method is the most sensitive to the sample size.

Fig. 7.

(a) Represents the RMSE results using three different sampling methods for quality evaluation when the sampling ratio in map A is 2%, 4%, 6%, 8%, 10%, 12%, 14%, 16%, 18%, and 20% respectively. (b)–(f) RMSE results of map B, C, D, E, and F, respectively.

SECTION V. Conclusion

Whether in the quality assessment, accuracy verification, or statistical investigation of RSGD, the stratified sampling method is economical and efficient [63]. The SSC method, which is stratified based on classifications of landcover elements, is the most widely used sampling method at present. However, the problem of this method is that the correlation between classifications of landcover elements and quality is not clear, and the improvement of sampling efficiency is limited.

This article demonstrates the feasibility of the proposed SSF method and presents the conceptual flow of the method. In the SSF method, the box-counting method is used to quantitatively characterize the fractal structure of elements in RSGD. The fractal dimension calculated by the box-counting method is used as the stratification indicator, the stratification boundary value is optimized by the cumulative square root of frequency method, and then the sample allocation is optimized by the proportional allocation method.

To validate the effectiveness of the SSF method, this article conducted an experiment using the South Sudan GCVD 2020 dataset across various spatial locations. Through the analysis of the experimental results, the following conclusions were drawn.

1. Based on DEFF, the SSF method designed in this article was compared with the SRS method. The experimental data analysis results showed that the DEFF values were less than 1, which verified the effectiveness of fractal dimension as a stratification indicator.

2. Based on RMSE, the SSF method proposed in this article was compared with the SSC method and the SRS method. The results showed that when the sample size was same, the SSF method could provide more accurate quality evaluation results. It proves that the SSF method is better than widely used the SSC method and the SRS method.

3. The SSF method proposed in this article is more sensitive to the sample size, especially small sample size, the performance of the SSF method is more significant, which proves that this method is more economical and efficient than the SSC method and the SRS method.

It should be noted that the proposed SSF method was tested exclusively on a single remote sensing dataset for the experiment conducted in this article. The multiple remote sensing datasets that vary in terms of bands and resolutions pose a challenge to the applicability of our method. Further research will concentrate on evaluating the representativeness of samples extracted using the proposed SSF method across multiple remote sensing datasets.

Furthermore, the proposed SSF method is currently only suitable for vector data. Further research will focus on whether the fractal dimension calculation in our proposed method is suitable for raster data, whether the element-based representation of samples works well with raster data, and whether the samples extracted by the proposed method are representative of the quality information of raster data.

Finally, the experimental study of the proposed SSF method in South Sudan is helpful to the global RSGD quality assessment.