Effectiveness of Super-Resolution Technique on Vegetation Indices

The identification and interpretation of remote sensed (RS) objects in an image depend on how well the sensor captures the region. In rare cases, RS images may be vulnerable to a lack of interpretability issues in some parts of the image due to the sensor's limits and preprocessing techniques. Conventionally, the interpretation of the land cover pattern's shape and size is apparent when the distance between the sensor and object is closer to the visualization level of objects and viable with digital airborne imagery. In this paper, an integration of the Super-Resolution (SR) technique in the high-resolution imagery to achieve the closer visualization level for mapping the vegetation is proposed. This approach enables the higher interpretive potential to define the land pattern's shape and size with very high spatial resolution, closer proximity, detailed and distinguishable patterns. This approach helps to precisely predict the total vegetated study area for land use and land cover changes (LULCC) and chlorophyll-rich vegetation applications. The proposed algorithm is carried out in two phases. In the first phase, the SR technique applied test images are tested for vegetation detection and mapping the vegetation information in a region with fourteen vegetation indices. In the second phase, similar testing is done without applying SR for input test images. The experiment revealed that test images using the SR technique yielded higher average values of 5 percent and 1 percent for the Normalized Difference Vegetation Index (NDVI) and Fractional Vegetation Cover (FVC), respectively, as compared to images tested using the non-SR technique.

SECTION I. Introduction

Vegetation detection using satellite images is critical in remote sensing applications due to climatic barrier resistance, ecological balance applications, and a requirement of the high level of interpretability imagery from high-resolution sensors, etc., [1]–​[3]. Land cover classification, emphasizing chlorophyll-rich vegetation identification, is critical for monitoring and planning urban development, autonomous navigation, drone mapping, biodiversity conservation, and other applications. The vegetation detection methods were used on RGB and near-infrared (NIR) bands in high-resolution airborne color images with the application of applying the normalized difference vegetation index (NDVI) for vegetation detection [4]. However, the leaves’ properties, the stems, other canopy components, and the background soil depend on vegetation canopies’ spectral signatures. Moreover, the conditions of solar illumination and viewing angles used by the satellite sensor play a critical role. If all these conditions were the same at a different position globally, then the spectral signature would be similar. When the study area comprises complex vegetation forms/ different vegetation stages, it is difficult to analyze the vegetation when spectral variations are similar [5].

Measurements of reflectance can help to expose the mineral content of rocks, soil moisture, vegetation health, buildings’ physical composition, etc. These measurements require the images from many spectral bands (red, green, blue, NIR, SWIR, thermal, etc.), the width of each band, and certain spectral bands (or combinations) that are good for identifying specific ground features. This requirement is feasible for Multispectral (MS) and Hyperspectral images. When working with high-resolution color images, vegetation detection computation can be accurate when specific distinct structures represent vegetation [5]. There are two solutions to extract and detect vegetation features accurately- (i) using classification algorithms and (ii) increasing the distinct possibilities through high-resolution imagery. In the first case, classification algorithms require robust supervised/ unsupervised classification techniques with many trained data sets in supervised algorithms and strong inventory measures to define the classification labels, which is a time-consuming process [5]. There is a requirement of ancillary data, including field samples, topographical characteristics, environmental characteristics, and other digital layers of data (geographic information system) for enhancing the accuracy of classification. It is desirable to carry out a vegetation classification using data collected from the same sources and at the same period and to apply the same processing algorithms to the whole area. In vast regions, the lack of reliable and similar data (remote sensing data and reference data) also limits high-quality vegetation maps. Hence, improving the quality of high-resolution imagery in the second case is feasible, compromising the high cost of imaging solutions [5].

The spatial resolution is the smallest spatial area captured by the sensor, which accurately describes the geometrical quality and objects. Spatial resolution is measured in Ground Sampling Distance (GSD) which is the distance between two neighboring pixels. When working with an ideal sensor, its GSD is equal to the inverse of the Nyquist frequency for that detector array [6]. The spatially resolved descriptors refer to the degree of detail that a sensor discerns [6]. The guidelines for spatial resolution [6] are (i) Coarse-resolution (pixels are defined at GSD in 30m or greater), (ii) Medium resolution (GSD in 2-3m), (iii) high resolution (GSD in 0.5-2m) and very high resolution ($< 0.5\text{m}$ ) [6]. The selection of images from the mentioned spatial resolutions plays a crucial role in vegetation cover as driven by (i) mapping objective, (ii) cost of the images, and (iii) image interpretation issues. Usually, a very high resolution ($< 0.5\text{m}$ ) has a higher potential for higher interpretability. Still, it limits imaging sensors’ cost, which is not feasible without digital airborne imagery [6].

The SR technique [7]–​[14] was used to improve the efficacy of distinguishing capabilities in high-resolution images by populating closest sampling points in the imaging systems’ sampling grid, thereby facilitating visually pleasing images. In terms of pixel density, this can be achieved by resolving events at different times. The increase in spatial resolution is achieved by increasing the number of pixels per unit area in an image. As a result, the image will exhibit sharp edges at corners and enhanced spatial features [7]–​[14]. Integrating the SR technique in the high-resolution imagery achieves the closer visualization level for the vegetation mapping and land use and land cover change (LULCC) applications.

In the organization of the paper, Section II intricates the contributions of the present study and its significance. Remotely sensed image considered for evaluation is mentioned in Section III. Mathematical modeling of the vegetation detection algorithm is discussed in Section IV, which illustrates the algorithmic design of filter banks, bias values, etc. In Section V, the phase-1 and phase-2 results are critically examined and analyzed. Based on the analyses and interpretation of base facts, Section VI concludes the proposed work.

SECTION II. Contributions of the Present Study and its Significance

A. Motivations of the Proposed Technique

The motivations for proposing this algorithm are:

1. Conventionally, optical systems render higher spatial resolution with accuracy in land cover maps. However, the optical data (either multispectral or hyperspectral) requires a detailed thematic map, a time-consuming process with complex algorithmic modeling, and extensive training data sets. A higher spatial resolution land cover map with good thematic content would help LULCC [15]–​[30].

2. The stereo photographic image is treated as a cost-effective planning tool owing to the expertise for visualizing and planning the landscape measurements and mapping the LULCC.

3. Visual inspection of the land in inaccessible areas may have insufficiently detailed representations. A significant amount of measurement data is needed for a good-quality description of such positions, which cannot be obtained without direct contact with the object being measured. Digital Surface Model (DSM) helps in providing detailed representations about the terrain. This visualization requires fieldwork and reference data collection on pre and post-assessment environments for terrain mapping vegetation detection.

These reasons drive to model an image processing algorithm for high-resolution color images that will increase the image’s spatial resolution and make visible RS objects accurately validate vegetation detection close to the geological inventory report.

B. Contributions of the Proposed Technique

The proposed work is carried out in two phases, which is depicted in Fig. 1 are:

FIGURE 1.

Block diagram of the proposed technique. Phase-1 (Indicated in black dashed arrow mark) implements vegetation detection and metric calculation using the SR technique. Phase-2 (Indicated in red dashed arrow mark) implements vegetation detection and metric calculation without using the SR technique. Simultaneously, comparing the SR image with the input image, the SR exhibits sharp edges, clear patterns, etc. The significant analysis at each step is showcased in this figure.

Phase-1: Step1: Loading the test image: The publically available high-resolution color test images are tested in this experimentation. The test images consist of a vegetation region covered with other signatures like buildings, roads, rural, urban, etc. These images are aided with Normalized Difference Vegetation Index (NDVI) range values in a Geological Inventory report computed by the Geologist with either fieldwork or a survey in [28-31].

Step2: SR algorithm application: The test image is tested with an SR algorithm.

Vegetation Detection algorithm (Step-3 to Step-5):

Step3: Decorrelator stretch: The color differences for the test image is highlighted using Decorrelator stretch for the vegetation detection map.

Step 4 and 5: NDVI and Fractional Vegetation Cover (FVC) calculation: NDVI with FVC and thirteen other vegetation indices are calculated. This phase’s outcome is the vegetation detected map of a region with an NDVI value, which can be cross-correlated with the values reported in the geological inventory report of respective test images.

Phase-2: The phase-2 is the replica of the phase-1 process, leaving step-2 in phase-1. i.e., Step-1, 3, 4, and 5 are carried out, where the test image without SR application on the image. This phase’s outcome is the vegetation-detected map of a region with an NDVI value and thirteen vegetation indices that can be referenced to validate the proposed technique’s effectiveness.

On experimenting the phase-1 and 2, the SR algorithm’s effectiveness on the vegetation indices can be visualized, which renders the additional thematic and vegetative information in a region accurately. The detailed contributions are explained in Section II-1 and Section II-2 next.

1) SR Experimentation for Test Image

For carrying out phase-1, the experimentation on state-of-the-art SR techniques for the test images^1–4 was performed. This incorporation renders finely grained spectral homogenous patterns in regular shapes to define vegetation regions accurately. The state-of-the-art of SR techniques considered for this experimentation is Super-Resolution Convolution Neural Networks (SRCNN) [7], Fast Super-Resolution Convolution Neural Networks (FSRCNN) [8], Efficient Sub Pixel Convolution Neural Network (ESPCN) [9], Deep Recursive Convolution Neural Networks (DRCN) [10], Laplacian pyramid (LapSRN) [11], SRResNet [12], MemNet [13] and Enhanced Deep Residual Super Resolution (EDSR) [14].

Table. 1 shows the type of network and its significance. It can be seen that extracting high-frequency cues renders coarsely grained feature representation information, and the analyses on five different network types help accurately analyze the progressive improvement of spatial information. Since this paper aims to extract the vegetation regions with high-frequency residues, Residual types of networks are desirable for these experimentations, although GAN has better perceptual quality. GAN’s desirability can produce more details, changing the patterns’ interpretation when there are sharp edges at high-frequency components. The combination of the sharpness at low-frequency components and preserving residues (Minute structural information) at high-frequency components is desirable for any residual networks.

TABLE 1 Significance of SR Techniques Used in This Paper

Table. 2 shows the parameters used in SR experimentation. From Table. 2, it can be seen that the combination of GRL and LRL offers a better super-solving capability [33]. The depth and filters are directly linked and enable the two entities to have a balanced tradeoff. The loss function depends on the dataset’s design, the filter forms, and the architecture used for SR [33]. In the training stage, two parameters of the algorithms [7]–​[14], momentum and weight decay, are set to 0.9 and 0.0001, respectively, to ensure an optimized learning rate [33]. These constant values validate the data on a specific ground scale to be equal, which provides a convenient comparison of the algorithms considered. In LapSRN and EDSR, the L1 normalization of layers is performed to reduce memory cost by maintaining the same amount of memory in the other convolutional layers.

TABLE 2 Parameters of SR Techniques Used in This Paper

The SR algorithms-SRCNN, FSRCNN, LapSRN, and EDSR are implemented in Matlab. Satellite images considered for this experimentation are trained with various scaling variables and, especially for the three upsampling variables, are loaded into the Matlab workspace for effective memory management. The mathematical equations are written in three stages as code; the first stage consists of coding the handling of input and output images.

The second stage requires the definition of variables for weights, bias, upsampling variables, etc. The third phase is coding the deep learning algorithms created by algorithmic mathematical derivations and inbuilt Matlab functions. For Eg. the commonly used Matlab function in the SR architectures mentioned above is the convolution operation “convolution2dLayer(filterSize,numFilters)”. The function convolves the input by moving the filters along the input vertically and horizontally and computing the dot product of the weights and the input, and then adding a bias term. The filtersize “filterSize” and the number of filters “numFilters” utilized is dependent on the SR architecture in the literatures. The upsampling factor testing at the three levels mentioned will facilitate insight into obtaining an output image of a higher resolution than that of the input with a higher level of details in the image. The images considered are the pan-sharpened version taken from the portal^1–4. The deep Convolution Neural Networks (CNN) algorithms- SR-Resnet, DRCN, Memnet, and ESPCN, are experimenting with using the respective authors’ publicly available python codes on GitHub website^5–8. These codes are utilized for this experimentation which is simulated in the Google Colab Notebook. The feasibility of appropriate upsampled images ($\times 2$ , $\times 4$ , $\times 8$ ) for the proposed technique is validated.

1. Qualitative analysis is done by human visual perception. The analyses on the SR capability in landscape coverage of features uncovered in the input image, sharp edges, and clarity in the pattern are observed.

2. Quantitative analysis is done in terms of

   • Reference-based metrics are typically calculated between two variables, i.e., Output and Input. The metrics such as Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), and Feature Similarity (FSIM) are used.

   • Non-Reference-based metrics are typically calculated for one variable, i.e., either/and input and output variables, separately. Non-Reference-based metrics such as Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Perception-based Image Quality Evaluator (PIQE) are introduced in this experimentation.

These upsampling testing variations would give insight into the appropriate level of interpretability in the pattern, giving accurate FVC. The experimental results reveals that EDSR algorithm is best suited for the test images considered. Hence, the EDSR applied test images at the upsampling factor ($\times 4$ ) is the vegetation detection algorithm, which will have the ability to identify small changes in vegetation land cover mapping.

2) Vegetation Detection and Indices Experimentation

For both phase-1 and 2, the test image is modeled into Color Infrared composite (CIR) and segregated into four different bands (NIR-Red-Green-Blue). The reason for choosing this conversion is because each sensor categorizes its wavelength for different characteristics such as Vegetation discrimination, soil moisture, mineral content, etc. The three bands are Near Infrared (NIR), visible Red, and Green band. Inherently, Multispectral and Hyperspectral images have these bands while capturing from the sensor. The test image considered is high-resolution color images. Hence, a code is created for this CIR conversion using pixel value mapping to convert to detect vegetation effectively. The differences in the three bands are used for validating the quality of the vegetation map. The spotted vegetated regions are validated quantitatively through the Normalized Difference Vegetation Index (NDVI) [1]. Thirteen other indices like ReNormalised Difference Vegetation Index (RDVI) [15], Transformed Vegetation Index (TVI) [16], Soil Adjusted Vegetation Index (SAVI) [17], Optimised Soil Adjusted Vegetation Index (OSAVI) [18], Transformed Difference Vegetation Index (TDVI) [19], Enhanced Vegetation Index (EVI) [20], Green Normalised Difference Vegetation Index (GNDVI) [21], Wide Dynamic Range Vegetation Index (WDRVI) [22], Visible Atmospherically Resistant Index (VARI) [23], Leaf Area Index (LAI) [22], Modified Non Linear Index (MNLI) [24], Modified Soil Adjusted Vegetation Index (MSAVI) [25] and Atmospheric Resistant Vegetation Index (ARVI) [26] are analyzed for the region considered.

Table. 3 illustrates the Vegetative Indices descriptions with their optimal range values and corresponding equation. Table. 3 shows the various indices that help quantify NDVI analysis of remotely sensed images from various viewpoints, such as vegetation quality, soil pixels, atmospheric effects, and the dynamic range of vegetation regions and vegetation in urban areas. These characteristics are grouped into two-(i) vegetation defining indices and (ii) Impact of atmospheric effects on vegetation. NDVI [1], RDVI [15], TVI [16], TDVI [19], and GNDVI [21] indices contribute to vegetation detection. SAVI [17], OSAVI [18], EVI [20], VARI [23], LAI [22], MNLI [24], MSAVI [25] and ARVI [26] vegetation indices define the impact of soil, atmospheric effects and biophysical effects.

TABLE 3 Vegetative Indices With the Description and Optimum Range of Metric Values

The proposed technique’s output will be the FVC quantitatively, where the percentage of vegetation in the region is estimated. The proposed technique’s effectiveness can be cross-correlated with vegetation detection in the region without an SR technique.

C. Significance of the Proposed Technique

The distinctiveness of the proposed technique from the conventional techniques can be elucidated in two forms- (i) data used and (ii) methodology adopted.

1) Data Used

It is seen that the works of literature [2], [37]–​[45] use multispectral, hyperspectral, radar, and optical images. The proposed technique uses pan-sharpened versions of the multispectral images, which are available in the portals^1–4. This data uniqueness indicates that the conventional vegetation detection/indices depend on the band difference and fusion of the bands in the images.

2) Methodology Adopted

While seeing the contributions in the closely related literature works [2], [37]–​[45] to this work, either single or combination of indices (max. up to eight indices) are used to validate the technique’s effectiveness for LULCC and vegetation detection applications. In this paper, fourteen vegetation indices, along with the introduction of FVC, are used for validation. Moreover, from the kinds of literature mentioned above, it is seen that the algorithms are mostly focused on spectral domain and there are no traces of spatial domain based SR application for vegetation detection and LULCC, highlighting the proposed technique’s distinctiveness.

SECTION III. Satellite Data

The test images considered for this experimentation are LISS-IV¹, Worldview-2², GeoEye-1³, and Spot-5⁴. Table. 4 shows details of the data are considered for this experimentation. The images considered are the pan-sharpened versions available in the portal^1–4. For Example, in Worldview-2², GSD of 0.46 cm for panchromatic image and multispectral layers at 1.84 meters of GSD have been pan-sharpened and shown in the portal. Furthermore, this imagery allows for the detection of trees’ crowns in the framework of precision agriculture, as demonstrated in [31], [32]. The data at various resolutions are analyzed for this experimentation to study the proposed technique’s impact on varied resolution images.

TABLE 4 Publically Available Dataset Considered for This Experimentation

The reasons for choosing the datasets in Table. 4 amongst the several datasets are

1. High-resolution satellite images defining precise agricultural change detection and mapping.

2. The data covers different signatures such as barren lands, vegetation region, and forest area, making it difficult to detect the vegetation map accurately.

3. The data are considered supplemented with the Geological inventory reports [27], [28], [29], and [30], where NDVI range values for the regions are available for validating the effectiveness of the proposed technique.

The publicly available images considered are unique by their characteristics, which remark the proposed technique’s effectiveness.

SECTION IV. Mathematical Modeling of the Proposed Algorithm

The proposed algorithm applies SR on the high-resolution color satellite image and visualizes its impact on the vegetation indices. This approach is compared without the SR technique vegetation indices. The mathematical modeling of the proposed technique is mathematically expressed in a step-by-step manner which is as follows:

Step-1-(Loading of Input Test Image): The input image in Fig. 1 is taken from the data mentioned in Section-2. EDSR, one of the SR algorithms, is utilized for vegetation detection.

Step 2-(SR Algorithm Application): Enhanced Deep Residual Super-Resolution (EDSR) [14] uses 32 residual blocks (Resblocks) with 256 convolution filters for the significant reconstruction of the HR image and uses two stages, which are as follows:

A. Feature Extraction

In this step, a set of feature maps are extracted directly from the test image. The extraction is done by cascading of convolutional neural networks at multiple scales. The mathematical formulation of 32 stages convolution is used with five numerous scales, which utilizes Eq. (1).

$$\begin{equation*} Y=\max (0,W_{m,n} \ast F_{m,n} (y)+B_{m,n})\tag{1}\end{equation*}$$ where $$\begin{equation*} W=[(Weight(F_{m} (x))-SpatialExtent]/(Stride+1)\tag{2}\end{equation*}$$

$F_{m,n}(y)$ is the convolution filter size where m varies from 1 to 16, and n ranges from 1 to 32; and $B_{m,n}$ is the n-dimensional bias vector. Spatial extent is filter size, in turn, depth of the filter. Stride is the number of spatial movements per pixel at a time, and Weight is transforming on input data within the network’s hidden layers.

B. Image Reconstruction

Image reconstruction reconstructs the high-resolution image from a coarser to a more visually pleasing manner. The mathematical formulation of the reconstructed high-resolution image utilizes Eq. (3).

$$\begin{equation*} X=(W_{m,n} \ast Y)+B_{m,n}\tag{3}\end{equation*}$$

For multiple-scale processing, the preprocessing module consists of two residual blocks of $5\times 5$ kernels with modules located at the head of networks intended to reduce the variance. The reason for choosing $5\times 5$ kernels is because of two factors:

1. A large number of pixels can be accessed simultaneously, which indicates a larger receptive field per layer.

2. The amount of information or features extracted are considerably less (as the next layer’s dimension reduces significantly), and the more significant amount of features can be taken out of the image.

At the end of the multi-scale model, scale-specific up-sampling modules are accessed in parallel to handle the multi-scale reconstruction.

The vegetation detection with its indices and FVC calculation are as follows:

Step 3-(Decorrelator Stretch): Decorrelator stretch computes the color differences between green, red, and near-infrared locations of the spectrum and highlights the difference. It stretches the imagery’s principal components by altering its distribution and utilizing the full range of 0–255 display devices. Since there is a lot of consistency in the data since many data sets have strongly correlated variables, it could be possible to construct fewer variables that contain all of the data in the data set while requiring less storage and processing power. The resulting images have the potential to be “better” than the originals. The principal components of the imagery method do the following (i) Extracting the eigenvalues from a covariance matrix, (ii) orders the eigenvalues and computes the percentage of the variance explained by each eigenvector, and (iii) computes the loading of each band on each of the principal components. The sum of the squares of the loadings on the principal components equals 1. It calculates the mean and variances of each band and displays the color differences. Decorrelator stretch computed by each correlation value normalized concerning the geometric average of color bands variances, which is given as

$$\begin{equation*} R_{ij} =\frac {R_{ij} (P,Q)}{\sqrt {R_{ii} (0,0)R_{jj} (0,0)}}\tag{4}\end{equation*}$$ where R_ij is the correlated value of each band with P, Q pixel values in x and y coordinates.

Step 4 (Computation of NDVI): NDVI is used for computing vegetation index [1]–​[3]. NDVI examines the relationship in the spectral variability of vegetation discrimination. The more considerable difference between NIR and RED should be visible to categorize the vegetation region. NDVI is calculated by normalizing the difference between NIR and RED band [2], [3] and corresponds to different signatures.

$$\begin{equation*} NDVI=\frac {NIR-RED}{NIR+RED}\tag{5}\end{equation*}$$ where NIR is the Near-infrared band, and Red is a red band.

NIR comprises low-frequency radiation adjacent to red hues in the visible. The NIR is influenced by the reflection of sunlight and is less affected by atmospheric scattering. The red band is primarily absorbed by chlorophyll which characterizes the richness of the vegetation. Hence, healthy vegetation has a high digital number (DN) value in the NIR band and low in the red band. Consequently, it has a larger NDVI. NDVI is a high positive number (+1) for healthy vegetation.

Step 5 (Fractional Vegetation Cover (FVC): Fractional Vegetation Cover (FVC) is the ratio of the vertical projection area of aboveground vegetation on the ground to the total vegetation area. The Fractional Vegetation Cover is the percentage of vegetation (defined [35]) conventionally calculated using Eq. (6)

$$\begin{equation*} FVC=\frac {NDVI-NDVI_{s}}{NDVI_{v} -NDVI_{s}}\tag{6}\end{equation*}$$ where NDVI_v is a fully covered vegetation region, and NDVI_s is the NDVI of the bare soil.

Since fully covered vegetation ($NDVI_{v}$ ) or pure bare soil ($NDVI_{s})$ calculation is difficult to assess as this combination is available in semi-arid or tropical regions, which is not viable in the dataset considered. Moreover, the maximum or minimum NDVI index using temporal or spatial characteristics require human expert knowledge on the threshold [35]. Hence, two aspects were experimented with to validate the effectiveness of FVC. One is grading-based, and the other is using another vegetative defining metric- LAI [36]. Firstly, Grading NDVI value for different FVC values is done using the following equation,

$$\begin{equation*} FVC=(NDVI+0.5)\times 255\tag{7}\end{equation*}$$

The value obtained using Eq. (7) is graded into six levels, namely 5, 5-50, 50-100, 100-150, 150-200, and 200–250 which represents the FVC of six different percentage levels, namely, 0%, 20%, 40%, 60%, 80%, and 100% respectively. Secondly, the two-dimensional distribution of vegetation is defined by FVC. LAI (LAI is calculated using the equation in Table. 3, second row) is a physical quantity that describes the vertical distribution of vegetation [36].

$$\begin{equation*} FVC=1-\exp (-0.5\times LAI)\tag{8}\end{equation*}$$

The heterogeneity of pixels can trigger mismatches between the results obtained from different resolutions when using remote sensing data on a different scale to obtain FVC [35], [36]. The FVC obtained from high-resolution data differs from that obtained from low-resolution data, causing a scale effect. FVC estimation is based on leaf water content results in an FVC value representing Green Vegetation (GV) [35]. Hence, validating in Grading and vegetation-defining metric will have high-precision FVC estimation within a magnificent scope.For phase-1, Step-1 to Step-5 of the proposed algorithm is carried out for the test images^1–4. In phase-2, Step-1 and Step-3 to Step-5 are carried out for the test images (leaving Step-2) for validating the effectiveness of phase-1 results.

SECTION V. Results and Discussion

The results and discussion are presented in two subsections: Section V. (A) (Step-1 and Step-2) covers SR techniques and how EDSR is chosen among the other techniques considered. Section V. (B) covers the vegetation detection and its indices (Step-3 to Step-5) for both phase-1 and phase-2.

A. Experimentation Results for SR Techniques

1) Qualitative Analysis

The algorithms considered for this experimentation are SRCNN [7], FSRCNN [8], ESPCN [9], DRCN [10], LapSRN [11], SRResNet [12], MemNet [13] and EDSR [14] at various upsampling factors ($\times 2$ , $\times 4$ , and $\times8$ ). Conventionally, these algorithms were tested earlier for the real-world datasets (scenery, faces, buildings, etc.). With the inspiration from the successful results of real-world datasets, these techniques imparted for the remotely sensed images reported in [33] and [34]. The test images mentioned in Section-2 are tested with the SR algorithms. The phase-1 SR application to the test images will render information on the coarsest acceptable pixel level for accurate interpretability of the features. This experimentation’s input image has sensor resolution in GSD as 5.8m¹, 1.84m², 0.46m$^{\mathrm {3,}}$ and 2.5m⁴. While upsampling the image to factors of ($\times 2$ ), ($\times 4$ ), and ($\times 8$ ), the resultant GSD is calculated as (GSD = sensor resolution/upsampling factor). The resultant GSD obtained using the aforesaid calculation for ($\times 2$ ), ($\times 4$ ), and($\times 8$ ) are [2.9m¹, 0.92m², 0.23m³, and 1.25m⁴], [1.45m¹, 0.62m², 0.115m³, and 0.625m⁴] and [0.725m¹, 0.46m², 0.0575m³, and 0.3125m⁴] respectively.

The reason for this GSD calculation are:

1. For validating spatial coverage effectiveness in terms of the algorithm’s super-resolving capability, a sensor’s spatial resolution is defined by GSD or pixel size. GSD will render the landscape coverage of the image objects uncovered / not visible in input images.

2. The SR approach has unity with the digital airborne imagery (i.e., a drone or any flight) in image visualization. The lowering of flight height will have more excellent feature coverage. But this is not always feasible on the recorded data. Alternatively, this concept is closely attributed by upsampling an image using the SR technique where the closer visualization level is achieved, which is not covered in the high-resolution imagery.

3. The visual perception validation can have a reference of small cover areas in greater detail.

The image considered at this resolution will have either of the two characteristics:

1. A large-scale image suggests a bigger, more accurate size of the ground features. The area of field coverage shown in the picture is smaller than on more minor scales.

2. A small-scale image indicates a smaller, less accurate size of the ground features. The area of ground coverage seen in the photograph is more important than on larger scales. The visual interpretation of images in terms of GSD at more minor scales will clarify detailed features in the area. The upsampling level designated in terms of GSD will have ease of understanding on proposed technique efficacy.

In practice, a large-scale aerial photograph’s high-spatial-resolution is integrated with an ortho-rectified image’s geometric accuracy. This integration will render geographically meaningful and geospatially accurate information. Coping of vegetation mapping with the geospatial information image is critical while dealing with LULCC. The maximum size of a resolved object depends on the upsampling factor. The full scope of the object reflects on the footprint in the image.

Fig. 2 shows the comparison of different responses when tested with the LISS-IV test image¹ shown in Fig. 2(a). The algorithms were analyzed under three different upsampling algorithms; Upsampling factor-2 ($\times 2$ ), Upsampling factor-2 ($\times 4$ ), and Upsampling factor-8 ($\times 8$ ). Fig. 2 (b, e, h, k, n, q, t, and w), Fig. 2 (c, f, i, l, o, r, u, and x) and Fig. 2 (d, g, j, m, p, s, v, and y) are images tested at upsampling factors 2,3 and 4 respectively for the algorithms- SRCNN [7], FSRCNN [8], ESPCN [9], DRCN [10], LapSRN [11], SRResNet [12], MemNet [13] and EDSR [14].

FIGURE 2.

Qualitative analysis of different deep learning algorithms in application to test image¹ with its respective upsampling factors ($\times 2$ to $\times 4$ ). The image captured from the sensor has some degree of inclination. On a standard ground scale, this inclination, in turn, reflected in slightly inclined object orientation on the processing of the algorithms for the same region of interest.

a: SR Capability

When comparing Fig. 2 (b-y) with Fig. 2 (a), there is a landscape extent of footprint size in the image, which shows the resolvable features. The landscape extent of footprint size helps increase spatial resolution in the image, identifying patterns clearly from a resolvable feature. This resolvable feature representation indicates the super resolving capability of the image.

b: Edge Strength (Sharp Edges), and Clarity in the Pattern

The upsampling factor-2 in Fig. 2 (b, e, h, k, n, q, t, and w) and upsampling factor-8 in Fig. 2 (d, g, j, m, p, s, v, and y) images are found to be having an introduction of artifacts/deformation in the structure of patterns. The clarity in the patterns was missing in these cases. The images obtained using algorithms- SRResnet, MemNet, and EDSR shown in Fig. 2 (q) to Fig. 2 (y), have the better super resolving capability in terms of clarity with enhanced spatial features thick lines on the edges. This superiority result is compared to images obtained using SRCNN, FSRCNN, ESPCN, DRCN, and LapSRN (Fig. 2 (b) to 2 (p)). Fig. 2(c, f, i, l, o, r, u, and x) shows that the SR images show a better representation of features and landscape extent than the input Fig. 2 (a). This representation indicates the effectiveness of the SR algorithms. Fig. 2 (c, f, i, and l) have unclear feature representation, and Fig. 2 (o, r, u, and x) has visibility of the image features. Fig. 2 (o, r, and u) has almost similar performance in terms of the image’s quality. Fig. 2 (x) renders better clarity of the image featuring a thick line on the edges. The Upsampling-4 ($\times 4$ ) results in Fig. 2 (c, f, i, l, o, r, u, and x) were found to the best fit for the proposed technique.

The reasons for the qualitative improvement are as follows:

1. Using suitable bias and weights, SRCNN and FSRCNN have substantial high-resolution reconstruction. The low-resolution (LR) image in ESPCN is upscale to the high-resolution (HR) image at the very last stage. Thus, since small-size feature maps are used, the number of computations for the network can be decreased. Consequently, relative to FSRCNN, the efficiency of ESPCN is enhanced. Although SRCNN upsamples the input image with single bicubic interpolation, ESPCN offers more feature maps for upsampling.

2. DRCN used a sequence of convolutions and rectified deep recursive units to smooth the image by retaining the image’s minute information.

3. LapSRN utilizes feedback-based upsampling that includes better image fidelity and more protection of the edge.

4. SR-Resnet uses high-frequency components with dense feature extraction that performs better than LapSRN.

5. MemNet implements a memory block to mine persistent memory directly via an adaptive learning mechanism consisting of a recursive unit and a gate unit. In various receptive fields, the recursive unit learns multi-level representations of the current state, providing better clarification.

6. EDSR utilizes an active sub-pixel convolution layer that retains maximum high-frequency components in the image, in addition to the standard feedback-based cascading convolution and rectified units.

Upon successful experimentation, it is found that EDSR at upsampling factor-4 performs better when compared to other state-of-the-art algorithms considered. Since the vegetation signatures require a clear definition of the patterns for effective detection of the image patterns, the images (both the test images and SR applied images) are visualized at a certain level. Hence, the upsampling factor-4 ($\times 4$ ) was tested for the data(s)-Worldview-2², GeoEye-1³, and Spot-5⁴ whose results are shown in Fig. 3. Beyond the upsampling factor-4 ($\times 4$ ), the images have smoothened patterns. A landscape extent of image footprint size with resolvable features may look over zoomed or stretched at this level. This landscape image is treated as slight deformation in image structure and homogeneity, leading to a change of interpretability in the image.

FIGURE 3.

Qualitative analysis of EDSR in application to test image^{2, 3, 4} with its respective upsampling factors ($\times 2$ to $\times 4$ ). The image captured from the sensor has some degree of inclination. On a standard ground scale, this inclination, in turn, reflected in slightly inclined object orientation on the processing of the algorithms for the same region of interest. While viewing at 150% zooming level, the SR image, in contrast to the input image, is viewed to check the edge strength and clarity in the pattern. This analysis will give insight into minute information lost or added to the image.

Fig. 3 (a, c, f) shows the test images of Worldview-2², GeoEye-1³, and Spot-5⁴, respectively. The zoomed portions of the test images at 150% level are shown in Fig. 3 (b, d, g). At ($\times 4$ ), the test image zoomed versions in Fig. 3 (b, d, g) were found to have a slight blur and an unclear definition of agricultural patterns at the region of interest. When these types of images are tested for SR analysis, enhanced spatial features with a clear definition of patterns by removing blurs are viable. Fig. 3 (c, e, h) shows that there is an enhancement in the spatial features in the image when compared to input images Fig. 3 (b, d, g) with better clarity in the patterns and thick line on the edges while zooming a region of interest at 150%. This capability helps to grain the input data, identifying the patterns of maximum resolvable landscape elements accurately [33]. Hence, the visualization of EDSR at ($\times 4$ ) is checked and recommended for minute details preservation in the image.

2) Quantitative Analyses

The quantitative analyses shown above can be validated quantitatively in terms of numbers. Since qualitatively, upsampling factor ($\times 4$ ) renders better results; the quantitative verification is done in detail. The DL algorithms are quantitatively analyzed with Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), and Information Fidelity Criterion (IFC) reported in [7]–​[14]. In this paper, the metrics are categorized into two- Reference-based metrics and Non-Reference based metrics. In addition to PSNR and SSIM, an additional parameter FSIM in Reference-based is introduced to understand the feature similarity between the input and output. Blurs and Pattern clarity assessment is assessed through PSNR, BRISQUE, and PIQE. The edge Strength in terms of Structure Preservation metrics like SSIM and FSIM. Table. 5 shows the quantitative analysis of the SR experimentation with the metrics mentioned above.

TABLE 5 Quantitative Analysis of SR Techniques Tested for the Images Under Test at Upsampling Factor-4 ( $\times4$ )

a: Reference-Based Metrics

Reference-based metrics are the metrics calculated based on output calculated concerning the input.

i) Peak Signal to Noise Ratio (PSNR)

PSNR is the critical measure for super-resolving the image. PSNR reflects image quality strength.

PSNR is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects its representation’s fidelity. Mean Square Error (MSE) defines the amount of error in the image. Lesser the error, the finer details of the image, which is closer to the ground truth (the image considered for testing).

MSE is calculated as an average of the squared errors between test and estimated image, which is given as

$$\begin{equation*} MSE=\frac {1}{n}\sum \limits _{i=1}^{n} {\big(F_{i}} -F_{m,n(y)} \big)^{2}\tag{9}\end{equation*}$$ where $F_{i}$ is the input and $F_{m,n(y)}$ is the output image. $$\begin{equation*} PSNR=20\log \left ({{\frac {Max(X)^{2}}{MSE}} }\right)\tag{10}\end{equation*}$$

With less MSE, the maximum PSNR is made, thus introducing the maximum significant coefficients. Since the PSNR does not have a particular range for determining the finest quality, the maximum value is generally agreed in practice between the other algorithms to validate the quality visually interpretable in an image by qualitative analysis.

From Table. 5, there is an increase in PSNR values at upsampling factor-4 from the SRCNN algorithm to the EDSR algorithm. When the upsampling factor is increased, the PSNR values for all algorithms decrease significantly. The padding adds artifacts that degrade image fidelity, resulting in a decrease in PSNR values. An increase in PSNR will occur if the artifact’s strength is muscular. In SR techniques, it is ensured that the padding is not responsible for the increased PSNR in all algorithms. There is a modest improvement of 1% in PSNR (on average of all test images considered) between the algorithms. There is a 2.75 percent increase in PSNR when comparing SRCNN to EDSR (on average of all the test images considered). This marginal improvement is significant as it is ensured that there is no noise or artifacts added upon while processing the test images using the algorithms. Moreover, the image’s quality preserved on algorithmic processing is critical for any DL algorithm.

ii) Structural Similarity Index (SSIM)

Structural Similarity Index (SSIM) considers the homogeneity and phase coherence of the original image’s gradient magnitude and the reconstructed image based on the similarity in three aspects-structure, brightness and contrast.

$$\begin{equation*} SSIM=\frac {(2~\mu _{x} \mu _{y} +C_{1})(2\sigma _{xy} +C_{2})}{(\mu _{x}^{2} +\mu _{y}^{2} +C_{2})(\sigma _{x}^{2} +\sigma _{y}^{2} +C_{2})}\tag{11}\end{equation*}$$ where $\mu _{x}$ and $\mu _{y}$ represent the mean value of the original image and reconstructed image, $C_{1}$ and $C_{2}$ denote the brightness of the two images. $\sigma _{1}$ and $\sigma _{2}$ are the variances of two images.

From Table. 5, the highest value of 98.13% (on average of all the test images considered) is observed for EDSR, which indicates that 98.13% of the information remains undisturbed. MemNet and SRResNet algorithms perform closer to EDSR with negligible differences. This structural information retention indicates that these three algorithms can retain maximum structural information on applying the SR algorithm.

iii) Feature Similarity (FSIM)

The quality of structures after algorithmic processing is characterized by Feature Similarity (FSIM) [34]. The FSIM index for full reference Image quality assessment (IQA) is proposed because the human visual system (HVS) understands an image mainly according to its low-level features by Phase Congruency (PC). The reason for choosing this metric is that human subjective evaluation can achieve very high consistency, particularly for sharp edges and blurs. These sharp edges support the SSIM metric if the image has a distortion.

FSIM is expressed as

$$\begin{equation*} FSIM=\frac {\sum \limits _{x\in \Omega } {S_{L} (x)^\ast PC_{m} (x)} }{\sum \limits _{x\in \Omega } {PC_{m} (x)}}\tag{12}\end{equation*}$$ where $\Omega$ is entire spatial domain, $PC_{m}(x) =$ max(PC₁ (x), PC $_{2}(x))$ is the phase congruence to weight the importance of $S_{L}(x)$ . $S_{L}(x)$ is the overall similarity between the images $f_{1}$ and $f_{2}$ .

From Table. 5, the FSIM values range between 0.93 and 0.98. This range reflects the increasing trend of FSIM as similar to SSIM. The trend remarks the efficacy of SR algorithms in terms of maximum information retention in the presence of blurs. MemNet and SRResNet algorithms perform closer to EDSR with negligible differences that follow SSIM.

b: Non-Reference-Based Metrics

i) Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE)

The BRISQUE [32] uses image pixels to measure characteristics in the spatial domain using locally normalized luminance coefficients. In deciding a visually appealing natural image without distortion such as ringing, blur, or blocking, BRISQUE is incredibly competitive. Since BRISQUE is a non-reference-based metric, it is separately measured for the input image and the output image. BRISQUE has values ranging from 0 to 100. The higher BRISQUE value approaching closer to 100 indicates that the resultant image is free of distortions and artifacts. Although the range shown in Table. 5 does not approach 100, the significant improvement as compared to the input is taken as the vital measure.

From Table. 5, there is an increase in the BRISQUE value when compared to the input. This incremental fashion indicates the distortion introduction is minimal due to the algorithmic processing in the image’s quality. There is a marginal improvement of 1.5% (on average of all the test images considered) in BRISQUE between the algorithms. While considering SRCNN to EDSR, there is an improvement in BRISQUE by 9.25% (on average of all the test images considered). EDSR, MemNet, and SRResNet have a better visual interpretation and distortion-free edges for the images considered in this experimentation that validate the qualitative analysis.

ii) Perception-Based Image Quality Evaluator (PIQE)

PIQE score [33] calculates the distorted image’s quality due to blocking artifacts and Gaussian noise. By overlaying the spatial quality masks on the blurred image, the PIQE value visualizes them. The PIQE score ranges from 0 to100. The lower the value, the greater the independence of the image from blocking objects and noise. Typically, the range values 0-20, 21-35, 36-50, 51-80, and 81–100 indicate different quality scales, namely, “Excellent”, “Good”, “Fair”, “Poor”, and “Bad”, respectively [33], [34]. It is seen from Table. 5 (Column-6) that the input image has “Good” quality, and on applying SR algorithms, the image is elevated to the “Excellent” quality scale. There is a decremental trend of values compared to the input, which remarks the superiority of SR algorithms in bringing enhanced SR capability with visually pleasing appearances.

From Table. 5, it is interpreted that for the SR applied images at upsampling factor ($\times 4$ ), EDSR performs well, followed by MemNet, SRResNet, LapSRN, DRCN, ESPCN, FSRCNN, and SRCNN algorithms in terms of both reference-based and Non-reference-based metrics. Hence, the EDSR algorithm is used to analyze further the vegetation detection algorithm and its indices, which are explained below.

B. Experimentation Results for Vegetation Detection and its Indices

1) Qualitative Analysis

The vegetation detection algorithm starts with the combination of CIR composite (NIR-Red-Green-Blue) and decorrelator stretch, shown in Fig. 4 (a-d). CIR imagery consists of a mixture of colors within the visible spectrum with near-infrared (NIR) visible within the visible spectrum by another hue. The resultant image when an RGB image is fed as input will be NIR-Red-Green-Blue. The human eye can capture high-frequency electromagnetic radiation from only a tiny portion of the electromagnetic spectrum. CIR imagery uses the NIR electromagnetic spectrum component that lies well outside the color red’s visible wavelengths. Decorrelator stretch is used after the CIR composite improves the image visually by enhancing the color differences in an image. Decorrelation stretching increases an image’s color differentiation with significant band-band correlation. The exaggerated colors discriminate the features easier. The color intensities of each pixel are translated into the covariance or correlation matrix of the color eigenspace of the Number of Bands-by-Number of Bands, extended to equalize the variances of the unit, and then converted back to the original color bands. Fig. 4 shows that the entire possible vegetation region falls in the Near Infrared region. Each intensity value ranging between 0 and 1 corresponds to different signatures like Non –barren vegetation lands, medium vegetation, and low vegetation are highlighted in light greenish-blue, dark green, and light green colors, respectively. Similarly, urban, semi-urban, highway, and forest are highlighted as yellow, pinkish-red, Orange, and red colors, respectively.

FIGURE 4.

Decorrelator stretch- (a), (b), (c) and (d) of test image^{1,2, 3, 4}. The color map shows the range of intensity values between 0 and 1. A new set of color values with a wider range is mapped to the image’s original color values. Within each band, the transformed color range is mapped to a normalized interval between 0.01 and 0.99, saturating 2%.

Since it is difficult to spot the images between the SR and Non-SR applied to the vegetation cover map, the visualization of the effectiveness can be done quantitatively only. Hence, the SR applied test images tested for vegetation cover map and NDVI based vegetation indices assessment is showcased in Fig. 5–​Fig. 9. Fig. 5 shows the vegetation cover map using NDVI. The input image to this testing is the SR applied for the test images^1–4. The RGB-composites using decorrelator stretch are represented as NIR-Red-Green spectral band for a better visual representation of vegetation signature. Since the combination of the NIR-Red-Green spectral band represents the foreground image, the difference between the visible NIR and red from the foreground highlights the vegetation region using the proposed technique, shown in Fig. 5.(d). This result can be cross-correlated with the difference in images of Fig. 5 (a) and Fig. 5 (d), which can be seen for the accuracy in vegetation detection. The spatial composite images of NDVI are produced to differentiate green vegetation more easily from bare soils seen in Fig. 5(c).

FIGURE 5.

(a-c) shows the vegetation cover mapping using $(a)$ visible NIR and $(b)$ Visible Red for the test image¹. (d-f) shows the vegetation cover mapping for test image^2,3,4. The circles shown in orange color indicate the vegetation regions spotted visually using NDVI. The input image tested for the vegetation cover mapping is SR applied test images^1–4.

FIGURE 6.

(a-n) shows the visual analysis of vegetation indices [1], [15]–​[26] calculated used indices equations shown in Table. 3 for the SR applied test image¹. This visual analysis qualitatively validates the vegetation cover. While computing the indices using the equations in Table. 3, the resultant image would be represented in a binary image without any reconstruction of the original color map, as shown in Fig. 4. A binary representation helps to discriminate the vegetation regions with non-vegetation regions visually.

FIGURE 7.

(a-n) shows the visual analysis of vegetation indices [1], [15]–​[26] calculated used vegetation indices equations shown in Table. 3 for the SR applied test image². This visual analysis qualitatively validates the vegetation cover.

FIGURE 8.

(a-n) shows the visual analysis of vegetation indices [1], [15]–​[26] calculated used vegetation indices equations shown in Table. 3 for the SR applied test image³. This visual analysis qualitatively validates the vegetation cover.

FIGURE 9.

(a-n) shows the visual analysis of vegetation indices [1], [15]–​[26] calculated used vegetation indices equations shown in Table. 3 for the SR applied test image⁴. This visual analysis qualitatively validates the vegetation cover.

As a consequence of this difference, the results of the rest three test images^2–4 are shown in Fig. 5 (e-f). In general, each reference polygon’s statistical signatures, using the pixel values of feature layers (RS-based and DEM-based) inside polygons, will be calculated. Here, this work aims to estimate the SR technique’s effectiveness on vegetation detection, which requires the spots that define these signatures. The critical analysis on how the difference between the bands helps to define signatures effectively is done. Hence, the differences in the NIR-Red-Green spectral band are sufficient for the vegetation detection spots visually. This spotting analysis is aided by human expertise, which will give insight comparative of vegetation region spotted visually with the quantitative numerical results.

NDVI with the confidence of thirteen vegetation indices is calculated using the equations mentioned in Table. 3. The outcome of the vegetation indices for SR applied test images^1–4, which can be visualized qualitatively in Fig. 6–​Fig. 9 (a-n). Conventionally, MS images are widely used for NDVI indices calculation. Stoplight color maps are widely used in the MS case where the spotlight of differences in NDVI changes is emulated in the red-yellow-green palette to NDVI-processed imagery. This color map is more intuitive, with green signaling healthy and red highlighting hotspots of areas lacking vegetation. The input RGB image palette is expanded to NIR-Red-Green-Blue in the proposed algorithm, spanned between Blue (0) and light yellow (1), as shown in Fig. 4. NDVI is particularly useful when comparing crop health, in which a color ramp, usually red-to-green, reflects the relative NIR values; Red is unhealthy, and green is healthy. Values that do not have NIR (i.e., ground) reflectance or areas that are not important to the field of interest are transparent. Hence, the visualization of NDVI and its indices processed image will span for the range Black (0) and White (1). The spots in the image with healthy vegetation are seen in the dark black areas. Light black and white areas indicate those with no vegetation with barren land and barren land, respectively. It is crucial to visually observe the vegetation index changes obtained by high-resolution satellite image spectral analysis to control the vegetation cover map’s positive and negative dynamics. The difference in vegetation index dynamics indicates growth imbalances within the same culture or region.

As a consequence of different spectral ranges of remote sensing data, the vegetation indices in a particular pixel of the image are visualized in Fig. 6–​Fig. 9 (a-n). From Fig. 6–​Fig. 9 (a), vegetation regions of the remotely sensed image are highlighted in dark black color. Since the test images^1–4 is considered tested for vegetation in urban regions, the sparse vegetation regions are highlighted, indicating live green vegetation using this reflected light in the visible and near-infrared bands. Fig. 6–​Fig. 9 (b) looks thematically similar with less contrast to Fig. 6–​Fig. 9 (a) as both define vegetation regions. A lack of clarity in the edges is observed in Fig. 6 (b) and Fig. 8–​Fig. 9 (b). This lack of clarity of the image indicates that healthy vegetation is not emphasized. Fig. 7 (b) exhibits strong highlights of healthy vegetation where the vegetation regions are highlighted in dark black colors. Fig. 6 and Fig. 8–​Fig. 9 (c) support Fig. 6–​Fig. 9 (a) by highlighting the Vegetation regions in white color. Fig. 7(c) exhibits more vegetation regions that are highlighted with dark black color. Fig. 6–​Fig. 9 (d) looks similar to Fig. 6–​Fig. 9 (a), as there is no effect on soil pixels for the test images considered. Fig. 6–​Fig. 9 (e) looks thematically similar with less contrast to Fig. 6–​Fig. 9 (d), which identifies sparse vegetation where the soil is visible. No soil pixels are detected in Fig. 6–​Fig. 9 (e). Fig. 6–​Fig. 9 (f) highlights the vegetation cover in urban environments, which looks similar to Fig. 6–​Fig. 9 (e). Fig. 6–​Fig. 9 (g) highlights the impact of vegetation overcoming soil and atmospheric influences. Since the urban environments consist of signatures like buildings, roads, vegetation, river, etc., spotting the vegetation in these environments is crucial. Roads, borderline (streets), and barren lands are highlighted in white color, and sparse vegetation is highlighted in dark black color. The dark color portions indicate the sensitivity of data for some atmospheric conditions and dense vegetation areas. This distributed highlighting of signatures improves the vegetation signal in high biomass regions with improved sensitivity and improved vegetation monitoring by de-coupling the canopy background signal. Fig. 6–​Fig. 9 (h) highlights the green spectrum instead of the red spectrum in NDVI. Since decorrelator stretch replaces the Multispectral channels, the extreme red channel is difficult to visualize. GNDVI measure of the vegetation covers photosynthetic behavior used to determine the moisture content and nitrogen concentration in plant leaves.

The sparse vegetation regions, medium vegetation regions, and barren lands are highlighted in dark black, light black, and white colors, respectively, in Fig. 6–​Fig. 9 (h). Fig. 6–​Fig. 9 (i) highlights moderate-to-high vegetation density when NDVI exceeds 0.6. This measurement of near-infrared and red light levels using WDRI is at least three times more accurate than NDVI because it can collect subtle variations in the moderate to high vegetation density of the crop canopy significant for dense canopy crops and mature crops. Fig. 6–​Fig. 9 (i) is similar to Fig. 6–​Fig. 9 (a), which indicates NDVI is not greater than 0.6, and there is no moderate-high vegetation density observed. Fig. 6–​Fig. 9 (j) estimates the fraction of vegetation index with low sensitivity to atmospheric effects. To work with RGB data rather than near-infrared (NIR) data, VARI is tested to measure “greenness in an image”, and it is an RGB index for leaf coverage. The images in Fig. 6–​Fig. 9 (j) exhibit healthy vegetation in dark black color followed by white patches or dots. Conventionally, aerosol data is used in MS image band, giving LAI highlighted in green and orange color map. The aerosol data shows a few additional information as atmospheric effect components, excluding the image details’ watermarks and image logos. While comparing these images with the images in Fig. 6–​Fig. 9 (a), there is a visibility of more transparent features with sharp edges and contrast-enhanced versions. Fig. 6–​Fig. 9 (k) estimates foliage cover, forecast crop growth, and yield. Fig. 6–​Fig. 9 (k) exhibits strong edge structures with a high degree of contrast enhancement, and minute details in the image are highlighted more as compared to other images (Fig. 6–​Fig. 9 (a-j) and Fig. 6–​Fig. 9 (l-n)). The RGB image tested for LAI calculation resembles the output of an image that is generally indicated to show edge detection in the image. In Fig. 6–​Fig. 9 (k), vegetation is highlighted in dark black color and quantified within the crop region range. Barren lands and roads are highlighted in white color. The combination of white spots and black spots (excluding the image details’ watermarks and image logos) indicates the barren lands’ vegetation regions. Fig. 6–​Fig. 9 (l) highlights the strong relationship between vegetation indices and biophysical surfaces. This index improves the Non-Linear Index (NLI) to account for the soil history, integrating the SAVI.

While comparing the images in Fig. 6–​Fig. 9 (l) with Fig. 6–​Fig. 9 (d), the images exhibit dark contrast of images which demarcates the distinction between the barren with vegetation, barren and vegetation accurately with a difference in the color range between dark black, black and white with black patches. Hence, the soil portions indicated with white patches are seen explicitly in Fig. 6–​Fig. 9 (l). Fig. 6–​Fig. 9 (m) shows the soil range reduction and increases the dynamic range of vegetation regions. MSAVI is used to increase the limits of the application of NDVI to areas of high bare soil composition. In areas where indices such as NDVI or NDRE have invalid data, MSAVI is often used due to a small amount of vegetation or a lack of chlorophyll. Fig. 6–​Fig. 9 (m) highlights the objects more transparent and increases the signatures’ visual differentiation. The signatures highlighted in the images are barren land (white color), healthy vegetation (Dark black), and medium vegetation (white with black patches). As compared to Fig. 6–​Fig. 9 (d) and Fig. 6–​Fig. 9 (a), the minimized effect of the soil context and increased the dynamic signal range of the vegetation can be visualized clearly in Fig. 6–​Fig. 9 (m). Fig. 6–​Fig. 9 (n) shows the robustness to atmospheric effects for a vegetation region. Fig. 6–​Fig. 9 (n) supports Fig. 6–​Fig. 9 (g) and Fig. 6–​Fig. 9 (j) by highlighting the impact in the darker version of images. The vegetation analysis is not supported with aerosol data, which leaves difficulty in analyzing the image’s atmospheric effect. The patches of white dots (in Fig. 7 and 8(n)) and barren land (white color) is highlighted, excluding the image details’ watermarks and image logos. The highlights in the image indicate the minute sharp edge details.

2) Quantitative Analyses

The qualitative analysis is validated with the detailed analysis given below. The phase-1 results (SR applied test image subject to vegetation indices assessment) are validated with the phase-2 results (Test image subject to vegetation indices assessment), which are shown in Table. 6 and Table. 7.

TABLE 6 Quantitative Analysis of Vegetative Indices for With and Without SR Techniques Tested for Test Images1–4 With a Threshold at 0.4

TABLE 7 Quantitative Analysis of FVC(Fractional Vegetation Cover) for With and Without SR Techniques Tested for Test Images1–4

Table. 8 shows the variation of the NDVI concerning threshold. The interpretation of the threshold affects the vegetation detection percentage. There is an incremental change of the NDVI by 1.33 % approximately as the threshold increases. From Table. 8, as the threshold increases up to 0.4, the vegetation percentage increases. Beyond the threshold point of value 0.4, the vegetation percentage decreases by 0.9%.

TABLE 8 Impact of NDVI Threshold Variation on NDVI on SR Application for Test Image1

The significant threshold for this case is 0.4. Since this trend follows for the rest three datasets, the threshold of 0.4 is set, and vegetation metric calculation is performed based on this threshold. This effective threshold is subject to SR and non-SR-based vegetation metric assessment, which can be seen in Table. 6 and Table. 7. This quantitative effectiveness aided in visualizing Fig. 6–​Fig. 9 (a-n) at a threshold of 0.4 accurately. Hence, the vegetation indices results at threshold 0.4 are showcased in Fig. 6–​Fig. 9 (a-n).

Table. 6 shows the quantitative analysis of Vegetative indices with and without SR techniques tested for test images^1–4 with a threshold at 0.4. The statistical NDVI from NDVI inventory statistics report in [27]–​[30] ranges for the test images$^{\mathrm {1~to 4}}$ is (0.19-0.27), (0.2-0.4), (0.2-0.4), and 0.28, respectively. With this range as a reference, the phase-1 results are compared with the phase-2 results for SR’s effectiveness on vegetation indices.

C. Phase-1: With SR Technique Applied to Test Image at Upsampling Factor-4

The SR applied test image is subjected to a vegetation detection algorithm, which begins with modeling the test image to show the color differences using Decorrelator stretch. The vegetation detection map is computed from the color differences. NDVI with the thirteen vegetation indices is calculated using the equations mentioned in Table. 3. The quantitative results of all the test images’ indices are shown in Table. 6, Column-4 (With the SR technique applied to test images at upsampling factor-4). Table. 6 shows the vegetation metric analysis, which is carried out in two ways; (i) Vegetation defining indices and (ii) Impact of atmospheric effects on vegetation.

1) Vegetation Defining Indices

NDVI [1], RDVI [15], TVI [16], TDVI [19], and GNDVI [21] indices contribute to vegetation detection. The NDVI obtained 0.2132, 0.2873, 0.3540, and 0.2770 for the test images^1–4, respectively, follows the NDVI inventory statistics report in [27]–​[30]. The images captured from different resolutions impact differences in the NDVI vegetation indices range values. TVI is a changed version of NDVI to prevent negative NDVI values from running [16]. The TVI value is calculated by adding 0.50 to the NDVI value and taking the result’s square root [16]. There is no technological difference between NDVI and TVI in output image or active vegetation detection [16], [19], and [46]. Theoretically, the ratio values less than 0.71 are taken as non-vegetation, and the vegetation area is given a value greater than 0.71. Since all the test images considered are in the range above 0.7, which is evident from Table. 7, the vegetation region is highlighted more, which can be qualitatively visualized in Fig. 6–​Fig. 9 (c). To strengthen the consistency between the TVI and NDVI, the FVC is calculated. Since the test images^1–4 have vegetation regions in urban environments, a null result is observed for RDVI and TDVI. GNDVI has a 96% on the average green spectrum for all the images.

2) Impact of Atmospheric Effects on Vegetation

SAVI [17], OSAVI [18], EVI [20], VARI [23], LAI [22], MNLI [24], MSAVI [25], and ARVI [26] vegetation indices define the impact of atmospheric effects and biophysical effects on the vegetation. From Table. 7, it is evident that there is no effect on soil pixels on vegetation detected regions which leaves the indices SAVI, MNLI, and OSAVI to remain zero. Since around 5% and 11% of atmospheric effects remain undefined by the remotely sensed image for EVI and VARI, respectively, these indices show the value of 95% and 89% (on an average of all the test images) of vegetation resistance in the remote-sensed image. The EVI exhibit 95%, which is in the range of healthy vegetation. VARI estimate 89%, which indicates that the fraction of vegetation in an image with low sensitivity to atmospheric effects is 89%, and 11% reduction may be due to a lack of clear definition of channel characterization. The ARVI metric obtained is 0.7372 (on average, all the test images) fall within the range closer to 1 for a vegetation region. The indices- LAI and MSAVI define foliage crop effects and the vegetation region’s dynamic range, which indicates a low-medium vegetation region without any crop foliage effects [47]. LAI and MSAVI exhibit values of 3.233 and 0.9269 (on average, all the test images), closer to the range of 3.5 and (0-1), respectively. All the 13 vegetation indices mentioned above support the effectiveness of the NDVI metric.

D. Phase-2: Without SR Technique Applied to Test Image and Comparison With Phase-1 Results

The experimentation is similar to phase-1 but without SR application for the test image. The analysis in this phase is carried out in comparison with the results of phase-1.

1) Vegetation Defining Indices

NDVI [1], RDVI [15], TVI [16], TDVI [19], and GNDVI [21] indices contribute to vegetation detection. Like phase-1, NDVI obtained for the test images^1–4 follows the NDVI inventory statistics report in [27]–​[30]. From Table. 6, the values obtained in phase-1 (SR applied test image) have 5 % (on an average of all the images) more than the results obtained in phase-2 (Without SR applied test image). The TVI value for all the images is greater than the theoretical value of 0.71, indicating healthy vegetation. On average, TVI for phase-1 improved 11.63% compared to phase-2 results. Since the test images^1–4 have vegetation regions in urban environments, a null result is observed for RDVI and TDVI, which follows a similar trend with phase-1 results. GNDVI has a 93.65% on the average green spectrum for all the images. On average, GNDVI for phase-1 (SR applied test image) has an improvement of 2.58% compared to phase-2 results (Without SR applied test image). The vegetation-defining indices indicate a substantial improvement of 5%, 11.63%, and 2.58% for NDVI, TVI, and GNDVI, respectively, for phase-1 (SR applied test image) as compared to phase-2 results (Without SR applied test image). This substantial improvement indicates the SR technique’s effectiveness for the test images, identifying vegetation patterns precisely.

2) Impact of Atmospheric Effects on Vegetation

SAVI [17], OSAVI [18], EVI [20], VARI [23], LAI [22], MNLI [24], MSAVI [25], and ARVI [26] indices define the impact of soil, atmospheric effects and biophysical effects. From Table. 6, it is clear that there is no effect on soil pixels on vegetation detected regions. SAVI, MNLI, and OSAVI remain zero, which follows a similar trend as phase-1 results. Since around 9.75% and 10.38% of atmospheric effects remain undefined by the remotely sensed image for EVI and VARI, respectively, these vegetation indices show the value of 90.25% and 89.62% (on an average of all the test images) of vegetation resistance in the remote-sensed image. The EVI exhibit 90.25%, which is in the range of healthy vegetation. On average, EVI for phase-1(SR applied test image) has an improvement of 2.66% compared to phase-2 results (Without SR applied test image). VARI estimate 89%, which indicates that the fraction of vegetation in an image with low sensitivity to atmospheric effects is 89.62%, and 10.38% reduction may be due to a lack of a clear definition of channel characterization. On average, VARI for phase-1(SR applied test image) has an improvement of 2.66% compared to phase-2 results (without SR applied test image). The ARVI metric obtained is 0.7683 (on average of all the test images), which falls within the range closer to 1 for a vegetation region. On average, ARVI for phase-1 has an improvement of 4.21% compared to phase-2 results. LAI and MSAVI exhibit values of 3.1473 and 0.9125 (on average, all the test images), closer to the range of 3.5 and (0-1), respectively indicates a low-medium vegetation region without any crop foliage effects. On average, LAI and MSAVI for phase-1 have an improvement of 2.76% and 1.57%, respectively, compared to phase-2 results. From the vegetation defining indices, it is interpreted that there is a substantial improvement of 2.66%, 2.66%, 2.76%, and 1.57% for EVI, VARI, LAI, and MSAVI, respectively for phase-1 (SR applied test image) as compared to phase-2 results (Without SR applied test image). This substantial improvement indicates the effectiveness of the SR technique necessary for the test images, identifying vegetation patterns with atmospheric effects precisely.

All the 13 vegetation indices mentioned above support the NDVI indices effectiveness with an improvement of 4.12% (on an average for seven indices excluding six null result indices) for phase-1 compared to phase-2 results. This improvement in vegetation indices for phase-1 remarks the effectiveness of SR on the test images for clear vegetation patterns for better interpretability and indices calculation.

E. FVC Calculation for Both Phases

Apart from the fourteen vegetation indices, including NDVI, FVC is introduced in this paper. FVC represents the extent of the photosynthetic plant region, the density of vegetation growth (tracking grassland status), and the vegetation growth pattern to some extent [48]. In the satellite-based FVC calculation, the coarse spatial resolution can create confusion, drawing attention to developing a rigorous statistical test for the relationship between field FVC and indices of satellite-derived vegetation [48]. With the application of SR, it is practically viable to interpret the vegetation region’s landscape extent accurately.

Table. 7 shows the FVC calculation in two aspects, which are experimented with to validate the effectiveness of FVC. One is grating-based, and the other is using another vegetative indices- LAI [34]. In the Grading method, FVC using NDVI value is calculated as shown in Eq. (7). The result of FVC is graded into six different slabs-5, 5-50, 50-100, 100-150, 150-200, and 200-250, which represents the FVC of six percentage, namely, 0%, 20%, 40%, 60%, 80%, and 100%, respectively.

From Table. 7, the first two rows correspond to the grading method. The highest spatial resolution (0.4 m) among the test images considered is test image³, where FVC value 214.93, accounted for phase-2 results. On application SR, there is an improvement in FVC value by 1.32%. The second-highest spatial resolution accounted for spatial resolution (1.84 m) test image² is 95.48 in phase-2 results. On application SR, there is an improvement in FVC value by 2.7%. Followed to that, the third-highest spatial resolution (2.5 m) test image⁴ subject to FVC obtained a result of 194.08. The corresponding phase-1 result (SR applied test image) accounted for an improvement of 2.08% in FVC value. Followed to that, the spatial resolution (5.8 m) test image¹ subject to NDVI obtained a result of 179.64, and the corresponding phase-1 result (SR applied test image) accounted for an improvement of 1.23% in FVC value. These improvements in value help understand that the closer the proximity between the object and sensor (GSD), the more accurate the estimation of FVC. The increase in GSD using SR helps to achieve better estimation instead of increasing the sensor resolution. While mapping the FVC obtained using NDVI value in Eq. (7), the Grading is done. The FVC value obtained using phase-1 is 199.63 (on average of all the test images), and using phase-2 is 196.03. There is an improvement of 1.83% of FVC grading while applying SR. The main limitation of Grading based method is the overlapping of all slabs for mapping FVC. E.g., Slab-2 (50-100) and (100-150) overlap where the resultant FVC value is 100. This overlap of slabs will significantly impact the determination of FVC for the region considered. Hence, the two-dimensional distribution of vegetation using LAI [34] defined in Eq. (8) is calculated.

FVC calculation using vegetative indices- LAI is shown in Table. 7 (Row-3). The values roughly vary between 0.78 and 0.81, which shows the healthy vegetation spotted in the regions considered. The average values of FVC for all the test images in phase-1 and phase-2 are 0.8013 and 0.7924, respectively. There is an improvement of 1.11% of the FVC value while applying SR.

From FVC calculation, the following conclusions are inferred-

1. When there is a decrease in spatial resolutions from the sensor, there is an improvement in NDVI and FVC estimation. For Eg, comparing the test image¹ and test image³ captured with 5.8 m and 0.46 m, the NDVI and FVC are higher for 0.46m than for 5.8 m.

2. Alternatively, when there is a decrease in spatial resolutions using the SR technique, there is a substantial improvement in NDVI and FVC estimation compared to a decrease in spatial resolution. For Eg. Comparing the test image² captured with 1.8 m and the SR applied (at upsampling factor-4) for the test image² obtained with 1.84 m, the NDVI and FVC results are higher phase-1 than for phase-2. This effect results in enhanced spatial coverage and clear distinctive patterns obtained using the SR technique.

3. Moreover, across the thirteen vegetation indices, there is an increase in values when SR is applied. These improvement remarks for useful vegetation information are adequately interpreted when applied to increase the sensor resolution merely.

From the qualitative and quantitative analysis of the proposed technique, it is seen that while increasing the GSD through the SR technique, there is a landscape extent of image footprint size with resolvable features. This increase in GSD helps to grain the input data, identifying the patterns of maximum resolvable elements accurately [33]. These maximum resolvable landscape elements have a significant impact on the improvement of interpretable vegetation patterns. Moreover, the impact is seen with an improvement in FVC, NDVI, and thirteen vegetation indices compared to test images without the SR technique.

SECTION VI. Conclusion

The impact of the SR technique on the vegetation cover and its indices is proposed and critically analyzed. When the test image covering vegetation map captured from high spatial resolution imaging sensors is subject to SR technique, the resultant image will exhibit the following characteristics- (i) very high spatial resolution image with a clear definition of the land cover pattern’s shape and size, (ii) sharp edges and clear structures, (iii) landscape coverage capability with prominent features which are unnoticed/uncovered in the input image and (iv) distinguishable land patterns for identifying the vegetation patterns from the total vegetated study area accurately. The fourteen vegetation indices (including NDVI) have an improvement of 4.56% (on an average for eight vegetation indices excluding six null result vegetation indices) for phase-1 compared to phase-2 results. Based on the SR technique’s effectiveness in vegetation metric estimation, the SR technique application is recommended for the higher interpretive potential of precise LULCC and chlorophyll applications that enhances the minute image’s spatial features by resolving the vegetation objects’ maximum size. Hence, a very high resolution (coarser spatially resolved patterns) using the SR technique has a higher potential for higher interpretability of vegetation-defined areas overcoming the limitations of imaging sensors’ cost and aid of digital airborne imagery.

ACKNOWLEDGMENT

The authors thank NIT Puducherry, Karaikal, for rendering research facilities in this area. They would like to thank Bhuvan-An ISRO Geo Portal source and Satellite Imaging Corporation for the satellite images.

Endnotes

¹https://bhuvanapp1.nrsc.gov.in/imagegallery/bhuvan.html

²https://www.satimagingcorp.com/gallery/worldview-2/worldview-2-rakaia-river/

³https://www.satimagingcorp.com/gallery/geoeye-1/geoeye-1-inakadate-japan/

⁴https://www.satimagingcorp.com/gallery/more-imagery/spot-5/spot-5-ajka-hungary-before/

⁵https://github.com/twtygqyy/pytorch-SRResNet

⁶https://github.com/ghif/drcn

⁷https://github.com/wutianyiRosun/MemNet

⁸https://github.com/Lornatang/ESPCN-PyTorch