Reducing Moisture Effects on Soil Organic Carbon Content Estimation in Vis-NIR Spectra With a Deep Learning Algorithm

When estimating soil organic carbon using visible and near-infrared spectra measured in situ, the interference of soil moisture content (SMC) needs to be eliminated. The existing SMC removal methods are mainly based on spectral transformation, but they change the original form of the soil spectrum. In this article, a new deep-learning-based SMC influence removal network (MIRNet) is proposed to establish the relationship between the spectra of moist soil and that of dry soil. This method constructs a spectral extraction module with two 1-D ghost modules to extract soil spectral characteristics and a context extraction module with a two-layer dilated convolutional neural network to extract the context information of the spectra. Then, these extracted features are combined to reconstruct the SMC influence with a two-layer deconvolution using residual learning. Finally, a new loss function that combines spectral distance and spectral shape measurement (D-S loss) is proposed. The input of MIRNet is the moist soil spectra, and the output is the dry soil spectra. Black soil collected from Harbin and yellow-brown soil collected from Nanjing are selected as the research objects. The $ R^{2}$ reaches 0.703, 0.747, 0.907, 0.892, 0.866, 0.907, and 0.926, respectively, when using spectra processed by external parameter orthogonalization, orthogonal signal correction, support vector regression, convolutional neural network, deep neural network, denoising convolutional neural network, and MIRNet. Therefore, the proposed MIRNet achieves competitive results compared with these state-of-the-art methods.


I. INTRODUCTION
S OIL organic carbon (SOC) content is an important indicator to measure soil fertility [1]. Rapid and accurate monitoring of SOC is of great significance to soil resource surveys, precision agriculture, and soil digital mapping [2], [3]. Hyperspectral remote sensing technology, with its advantages of a wide spectral response range and high spectral resolution, can quickly obtain the fine spectral characteristics of surface soil. In recent years, the estimation of SOC content by using the soil reflectance spectral characteristics measured by spectrometers has become a difficult and important task in soil science [4]. At present, researchers have established a relatively complete theoretical system for estimating soil properties by visible and near-infrared (Vis-NIR) spectra under stable observation conditions for different soil types [5], [6]. However, whether static in situ observations or real-time dynamic observations are used in the field, factors such as soil moisture content (SMC), temperature, and soil surface roughness will all affect the acquisition of soil Vis-NIR spectra. Among them, SMC is the most important factor affecting hyperspectral measurement in the field. It can cover the spectral absorption characteristics of SOC, and result in low accuracy of SOC content estimation using Vis-NIR spectral data of in situ soil directly [7]. Therefore, it is necessary to eliminate the influence of SMC on Vis-NIR spectra to achieve real-time measurement of SOC content in the field. In recent years, researchers have tried various methods to achieve a more accurate estimation of soil properties by eliminating the influence of SMC on soil spectra. Some researchers study soil property estimation by looking for windows in spectral bands that are less affected by moisture. Wu et al. [8] found that within specific wavelength ranges in the NIR spectra, at 800-1400 nm, 1600-1700 nm, 2100-2200 nm, and 2300-2500 nm, the first derivative of the spectra seems insensitive to the moisture content of the soil samples. These observations suggest the potential of focusing on these regions to determine the SOC content without any interference from SMC. Some researchers have used SMC hierarchical modeling to study the estimation of soil properties under the influence of SMC. Notica et al. [9] and Hong et al. [10] both chose the soil moisture-based cluster method. They classified the soil samples by spectrum according to the normalized soil moisture index and the SOC content was predicted by subsection modeling.
Some researchers improved the similarity among spectra using spectroscopic preprocessing or transfer algorithms. At present, the direct standardization (DS) method introduced by Wang et al. [11] and piecewise direct standardization (PDS) were successfully used to remove the effects of SMC. Ji et al. [12] derived the DS transfer matrix, which characterizes the differences between field and laboratory spectra, and used it for the correction of field spectra. Orthogonal signal correction (OSC) is an optimization method proposed by Wold et al. [13], which enables the removal of systematic variation from field spectra that is orthogonal to the reference data [14]. Biney et al. [15] verified the effectiveness of the OSC across three different agricultural fields for both lab-dry and in-field spectra and obtained good estimation accuracy.
Another way to approach this problem is to remove the effects of SMC on spectral parameters from the calibrations. The most widely used is the external parameter orthogonalization (EPO) algorithm. This method was first proposed by Roger et al. [16] and applied to the removal of temperature as an influencing factor when spectral prediction of sugar content in fruits was made. Minasny et al. [23] extended this method to remove the influence of SMC on field spectral measurements. The prediction of SOC was successfully made by using the spectra after orthogonalization treatment [17]. Since then, there have been many studies on SMC influence removal using EPO methods [18], [19], [20], [21], [22].
However, all the aforementioned methods have some problems in dealing with the removal of the SMC influence. The method using impervious spectra loses a large amount of effective information in soil Vis-NIR spectra, resulting in low accuracy of the estimation results even though some external factors are excluded. The hierarchical modeling method requires ensuring a uniform sample quantity at different water content levels, and the accuracy of SOC estimation depends on the accuracy of grouping. Neither method eliminates the effect of SMC on soil spectra directly. The DS and PDS algorithms based on spectral conversion perform correction for the whole band, which easily leads to the problem of overcorrection when spectral curves are similar. For the most commonly used EPO and OSC algorithms, the original spectra need to be preprocessed, such as multiplicative scatter correction, standard normal variate, detrend, and first derivative. This process requires several experiments to determine the most suitable pretreatment method, which has relatively low efficiency. It will destroy the structure of the original spectra, so it is impossible to simulate the corresponding dry soil spectra from the wet soil spectra. Moreover, these two methods are based on the premise that SMC and SOC content are independent, so it is difficult to simulate the interaction between SMC, SOC content, and soil spectra [23]. In view of the problems of the aforementioned methods, this article tries to apply deep-learning-based methods in the field of removing the effects of SMC.
Deep-learning-based methods have shown their advantages in many research areas, such as image classification [24], natural language processing [25], speech recognition [26], and remote sensing [27]. In recent years, many deep-learning-based models, including stacked autoencoder [28], deep belief network [29], convolutional neural network (CNN) [30], recurrent neural network [31], deep neural network (DNN) [32], and residual network (ResNet) [33], have been explored for hyperspectral image processing. To fully extract the features in soil spectra that contain considerable redundancy, deep-learning-based methods have recently been proposed for the estimation of soil component content. With their powerful feature extraction capability, these methods can express the spectral signal effectively [34], [35], [36]. This fully demonstrates the potential of deep-learningbased methods to analyze the effect of SMC on the spectra of in-field moist soil. Deep-learning-based methods are also widely used in denoising problems that are similar to the removal of SMC influence. Such studies include desert seismic data denoising [37], [38], [39], image denoising [40], [41], [42], and speech enhancement [43], [44], [45].
In this study, an end-to-end moisture content influence removal network (MIRNet) is proposed for improving the SOC content estimation accuracy of soil with different SMC levels. The proposed MIRNet consists of two branches, namely, the soil spectral feature extraction module (SEM) and the soil spectral context information extraction module (CEM). In SEM, as it is not easy to obtain spectral data of soil with different moisture contents and the number of samples is limited, a 1-D ghost module is used to extract spectral features. Because of the fast feature extraction, independent learning, fast optimal solution, and accurate fitting of complex nonlinear mapping of a deep CNN [46], [47], it is used in the basic structure in this module. In CEM, due to the high spectral resolution of hyperspectral data, information redundancy exists between adjacent bands. Therefore, a dilated convolutional neural network (DiCNN) is used to learn spectral context information contained in nonadjacent bands. Furthermore, according to the idea of a ResNet [40], the SMC effect is directly studied as a residual (Res) from moist soil spectra to obtain better accuracy. Finally, a new loss function combining spectral distance and spectral shape measurement (D-S loss) is proposed to better promote the learning and optimization of this network. The spectral distance measurement is the standardized Euclidean distance. The spectral shape measurement is the proposed sliding correlation coefficient.
The main contributions of this article are summarized as follows.
1) A dual-network is designed and implemented to extract and remove the influence of environmental factors on soil spectra. By making full use of the information of adjacent and nonadjacent spectra, the network can effectively extract the influence of soil moisture on soil spectra. 2) The idea of ResNet is integrated into the network to study the influence of SMC on the moist soil spectra directly, rather than establish the relationship between moist soil spectra and corresponding dry soil spectra. The effectiveness of this improvement is verified by experiments. 3) We proposed a new loss function, D-S loss, in this article to balance the spectral distance and spectral shape difference, which improves the training accuracy by judging the similarity of two soil spectra more accurately. The rest of this article is organized as follows. Section II presents the preparation process of the experimental data used in this article in detail. Section III is a description of MIRNet. Experimental settings, analyses, results, and discussions with Vis-NIR spectral data prepared with multiple SMC levels are shown in Section IV. The discussion is given in Sections V, and finally, Section VI concludes this article.

A. Soil Sample Collection and Sample set Division
In this study, 200 soil samples with a depth of 0-20 cm were collected in 2021. Among them, 100 soil samples were black soil collected in Harbin, Heilongjiang Province, and 100 soil samples were yellow-brown soil collected in Nanjing, Jiangsu Province. After each soil sample collection, the soil samples were processed to remove small gravel, dry branches, fallen leaves, animal residues, and other sundries. After that, the soil samples were brought back to the laboratory for air-drying, grinding, and sieving (≤ 2 mm) [48]. After the aforementioned treatment, each soil sample was divided into two parts. One of which was stored in a glass jar for the determination of VIS-NIR spectral data. The other portion was used to determine the SOC content by the K 2 Cr 2 O 7 -H 2 SO 4 oxidation method after sieving with 60 mesh screening [49]. In this study, these samples were further divided into three nonoverlapping subsets. They are described as follows, and notations similar as to those in [50] were used. The summary statistics of SOC content in the three different sets are given in Table I. 1) Dry ground set (S 0 )-This set consisted of 50 samples to develop dry ground multivariate models for the estimation of SOC content. Samples in this set were scanned once under dry conditions. 2) MIRNet development set (training set) (S 1 )-This set consisted of 100 samples for MIRNet development. Each sample in this set was scanned 11 times: one scan under dry ground conditions and ten scans at ten different moisture levels (1100 scans in total). A detailed description of the soil rewetting procedure is provided in the next section. 3) Testing set (S 2 )-This set consisted of 50 samples for independent MIRNet validation and testing of the SOC content estimation model. The samples in this set were also scanned 11 times and under the same moisture conditions as S 1 (500 scans in total).

B. Rewetting Procedure and Vis-NIR Spectra Scanning
In this study, the soil gravimetric water content rate is used as SMC, and 11 SMC gradients (0%, 4%, 8%, 12%, 16%, 20%, 24%, 28%, 32%, 36%, and 40%) are designed to conduct spectral observation experiments on the S 1 and S 2 soil samples. Approximately 150 g of soil from each sample was placed in a Petri dish and scanned under the air-dried conditions first. After that, 4% SMC was added to each soil sample as a standard, that is, 6-g water was sprayed evenly on the soil. It was then quickly sealed to prevent moisture evaporation and left for 8 h to distribute moisture evenly in the soil. At this point, the moist soil spectral data were immediately measured, and the soil was reweighed. The water addition process was repeated until spectroscopic measurements of all SMC levels were completed. Although there was a small difference in soil moisture at different depths within the soil sample, such small variations do not have a significant impact on the measurement of spectral data, so it is ignored.
Two spectrometers with a halogen lamp that can provide parallel light were used to acquire Vis-NIR reflectance spectra from 321.37 to 2598.2 nm in this experiment. The spectral range of the visible spectrometer is 321.37-1104.83 nm (spectral sampling interval of 0.82 nm). The spectral range of the near-infrared spectrometer is 890.74-2598.2 nm (spectral sampling interval of 6.6 nm). A standard white plate was used as a white reference to convert radiometric digital numbers to reflectance. Each spectrum was an average of 20 instantaneous internal scans. In the process of acquiring soil spectral data, spectral range of 321.37-450 nm, 1000-1200 nm, and 2350-2598.2 nm were found with very low signal-to-noise ratios. Therefore, these parts of the bands were excluded from data analysis. Spectra were resampled with 10-nm intervals to reduce the dimensionality of the data. Finally, the Savitzky-Golay 11-point filtering smoothing method (polynomial order 2) was used to smooth and denoise the resampled soil data [51]. After a series of operations, the final obtained spectral range of the soil spectra are 450-1000 nm and 1200-2350 nm. The soil spectral dataset for analysis and modeling was obtained, in which 50 soil spectral data (50 samples) were obtained for S 0 , 1100 spectral data (100 samples, 11 SMC gradients) for S1, and 550 spectral data (50 samples, 11 SM gradients) for S 2 . Fig. 1 shows the average reflectance spectra of soil samples in S 1 and S 2 under 11 different SMC levels. As shown in Fig. 1, with the increase in the SMC level, the soil Vis-NIR spectral reflectance curves present a downward trend. This decline is based on a certain pattern. The soil reflectance spectra show reflective valleys in three bands (1410 nm, 1930 nm, and 2210 nm), which are the absorption peaks of soil water. The change in SMC also has the greatest influence on the reflectance spectra of these three bands. In addition, when the SMC reaches more than 24%, the range of spectral changes becomes decreases, because the soil tends to be saturated with water.

A. Framework of the Proposed Method
The framework of the proposed MIRNet method is shown in Fig. 2. This network has three core parts: a SEM composed of a 1-D ghost module, a spectral CEM composed of dilated convolutional layers, and a spectrum reconstruction module composed of deconvolutional layers. The input of the network is the spectrum of the moist soil. After that, spectral features and context information are extracted from the original spectral data through SEM and CEM, respectively. Then, the two extracted pieces of information are combined to reconstruct the effect of SMC on the spectrum through the spectrum reconstruction module. At this time, the obtained is the SMC influence spectrum contained in the moist soil spectrum. Based on the idea of residual learning (Res), the input moist soil spectrum is used to subtract the learned SMC influence spectrum, namely, the estimated corresponding dry soil spectrum is obtained. Finally, the estimated and measured corresponding dry soil spectra are used to calculate the D-S loss value for adjusting the weight and offset in back propagation.

B. Data Preparation Process
The main objective of this study is to eliminate the effects of SMC on soil spectra, which need to be validated by estimating SOC content. To better understand the use of these three datasets (S 0 , S 1 , and S 2 ) in these two phases, several definitions are elaborated as follows.
The function of S 0 is to establish the SOC content estimation model using its dry soil spectra. Let the dry soil spectra in S 0 D S 0 ∈ R N S 0 ×B , N S 0 and B represent number of the samples and the number of spectral bands, respectively. Therefore, a sample set of is constructed, and D i S 0 represents the spectrum of the ith sample. The corresponding label set is composed of the SOC content value of the samples, which is defined as y S 0 = {y 1 S 0 , y 2 S 0 , . . . , y N S 0 S 0 }. S 1 and S 2 are applied to establish MIRNet using their moist soil spectra and dry soil spectra. They are also used to evaluate the accuracy of the removal of the SMC influence. This evaluation is achieved by comparing the spectral similarity before and after removal and computing the accuracy of SOC content estimation using the soil spectra after removal. S 1 and S 2 are defined with reference to S 0 . Let the moist soil spectra in S 1 and S 2 M S 1 ∈ R (n×N S 1 )×B and M S 2 ∈ R (n×N S 2 )×B , N S 1 and N S 2 represent the number of samples in S 1 and S 2 , respectively, and n represent the number of SMC levels in S 1 and S 2 . Therefore, moist-soil sample sets of S 1 and S 2 are constructed as and M i S 2 represent the spectrum of the ith sample in S 1 and S 2 , respectively. Let the dry soil spectra in S 1 and To match the moist soil sample sets of S 1 and S 2 , the dry soil spectra in S 1 and S 2 are extended to D S 1 ∈ R (n×N S 1 )×B and D S 2 ∈ R (n×N S 2 )×B , respectively. Therefore, dry soil sample sets of S 1 and S 2 are constructed as represent the spectrum of the ith sample in S 1 and S 2 , respectively. These dry soil spectra sets are the corresponding label sets for the establishment of MIRNet. To verify the SOC estimation model, the corresponding label sets are composed of the SOC values of the samples in S 1 and S 2 , which are defined as

C. Spectral Feature and Context Information Extraction Module
The basic purpose of SEM and the CEM is to extract the influence of SMC implied in moist soil spectra. In the SEM, hierarchical spectral features are obtained by convolution calculation. The moist soil samples in S 1 are fed into the 1-D ghost module. The basic feature extraction structure of the ghost network is convolutional layer. These layers constitute the feature extractor. In the training process, the convolution kernels learn to obtain reasonable weights. A pooling layer is added after each ghost module to further control the number of parameters. The pooling layer can be regarded as a special convolution process that can reduce the parameters of the model. The convolutional layer can be defined as follows: is the ath feature map of the previous (l − 1)th layer, p l j is the jth feature map of the lth layer, and k refers to the number of input feature maps. The weight w l j and bias b l j denote the descriptions of the jth convolutional filter in the lth layer.
In the 1-D ghost module, a small number of convolutional filters are used to generate the feature maps. Then, the cheap operation of linear transformation is used to obtain the ghost feature maps. The linear transformation can be defined as follows: wherep l j is the jth feature map of the lth layer and Φ is a linear transformation. Finally, the real feature maps obtained by the convolutional layer and the ghost feature maps are combined to form a complete output.
In the CEM, dilated convolutional layers are used to capture context information from the separated spectral channels. There is high similarity between adjacent bands of soil spectra, so it is possible to learn redundant information by using a traditional convolutional layer. Therefore, the conventional approach is to enlarge the receptive field by increasing the size of the convolution kernel. However, this will increase the amount of calculation. Due to this limitation of the traditional convolutional layer, a dilated convolutional layer has been derived to obtain a larger receptive field without increasing the amount of calculation.
To illustrate the difference between the 1-D dilated convolutional layer and the traditional convolutional layer, a convolution kernel with a size of 3 is taken as an example. The process of the 1-D dilated convolution is shown in Fig. 3. The dilation rates of the dilated convolution in Fig. 3(a)-(c) are 1, 2, and 3, respectively.
In these figures, the yellow boxes are the equivalent convolution kernel sizes.The blue dot indicates the value of the corresponding position of the convolution kernel when the dilation rate is equal to 1. The white point in the yellow box represents the injected hole, and the value is equal to 0. From the perspective of the size of the receptive field, the receptive field increases with increasing dilated rate of the dilated convolution. From the perspective of computational complexity, compared with the standard convolution, in the case of the same receptive field (excluding the dilated rate equal to 1), the parameters required for the dilated convolution training are less than the standard convolution and the greater the difference between the two parameters with the increase of the dilated rate. The relationship between the size of convolution kernel r and equivalent convolution kernel r can be defined as follows: where d is the dilation rate. When the dilation rate is equal to 1, the dilated convolution result is consistent with the standard convolution results. When the dilated rate is equal to 2, the receptive field of dilated convolution with convolution kernel size 3 is equivalent to the standard convolution with size 5. Therefore, the expression of the receptive field is where R i is the receptive field of the ith layer, R i+1 is the receptive field of the (i + 1)th layer, and T i is the product of strides of all previous layers, which can be defined as follows: The size of the output feature map is not affected by the dilated convolution. Therefore, the receptive field can be increased by using dilated convolution, while the size of the output feature map can remain unchanged. This also means that dilated convolution can expand the receptive field to obtain the context information of the spectra and avoid the information loss caused by the pooling layer. When the moist soil spectrum of S 1 (M i S 1 ) is extracted by the SEM and the CEM, the spectral characteristics F s i S 1 and spectral context information F c i S 1 are obtained, respectively. Then, the two features are concatenated by the concatenate layer, and result in a multiscale feature F i S 1 as

D. Spectrum Reconstruction Module and Residual Estimation
After extracting multiscale features, the proposed MIRNet reconstructs the spectra through the spectrum reconstruction module. This reconstruction of the spectra is achieved by deconvolutional layers and upsampling layers. Since the proposed MIRNet adopts the idea of Res, through the spectrum reconstruction module, the influence of SMC on soil spectra is generated by F i S 1 . This learned influence of SMC can be expressed aŝ The basic idea of Res is to build a complex nonlinear mapping relationship between the moist-soil spectrum and the SMC influence on the soil spectrum (noise). This relationship can be defined as follows: where E i S 1 represents the measured value of the SMC influence on the soil spectrum. The output of the proposed MIRNet is the estimated dry soil spectrumD i S 1 . During the training process, we minimize the distance between the estimated dry-soil spectrumD i S 1 and the measured soil spectrum D i S 1 by a proposed D-S loss function. The D-S loss evaluates the similarity of two spectra from two aspects: spectral space distance and spectral shape. The D-S loss function is constructed as follows: (8) In this study, the distance between the soil spectra of samples is measured by the standardized Euclidean distance. Therefore, can be defined concretely as follows: where D i(k) S 1 are the reflectance values of the spectra of D i S 1 andD i S 1 in a certain band, respectively. And σ i S 1 is the corresponding variance.
The correlation coefficient r can characterize the similarity of shapes between two spectra. It can be defined concretely as follows: The proposed sliding correlation coefficient r K introduces the sliding coefficient K. The estimated spectraD i(k) S 1 move in the direction of decreasing and increasing K wavelength, the correlation coefficients with D i(k) S 1 are calculated in the wavelengths overlap region, and their average is taken as r K .

E. SOC Content Estimation
In this study, a CNN estimation (CNNE) method is proposed for establishing the SOC content estimation model. This estimation model is used to evaluate the effectiveness of the SMC influence removal model. The framework of the proposed CNNE method is shown in Fig. 4. A two-layer CNN is constructed to extract useful features from the soil Vis-NIR spectra.
Dry soil spectra in the S 0 set (D s 0 ) are used to establish CNNE in this study. The input of the network is the original spectra D s 0 . The output of the network is the predictive SOC content y s 0 . The estimated SOC content and the measured SOC content y s 0 are used to calculate the loss value for adjusting the weight and offset in back propagation. The loss function can be written as the following formula: In this study, moist soil spectra in S 2 (M s 2 ) are applied to MIRNet to obtain the estimated dry-soil spectraD s 2 . Then, M s 2 ,D s 2 , and M s 2 are used for CNNE to estimate the SOC content. The estimation accuracy of the SOC content can be used as an index to evaluate the accuracy of SMC influence removal methods.

A. Experimental Design
In the experiments, MIRNet development set S 1 , which contains 1500 moist soil spectra is split into two nonoverlapping subsets, including the training set and the validation set. Specifically, the samples in the training dataset are selected from all the labeled samples in S 1 by the stratified random sampling method. Four SOC intervals (less than 2%, 2 % to 3.5%, 3.5% to 5%, and more than 5%) are set for the samples in S 1 . Then, different proportions of samples are randomly selected from the four sections. The validation set consists of all the remaining samples. Here, the validation set is used to evaluate the performance of the model during the training process. The CNNE development set S 0 , which contains 50 dry soil spectra, is split into three nonoverlapping subsets, including the training set, the validation set, and the testing set. Specifically, 30 samples in the training dataset are randomly selected from all the labeled samples in S 0 ; 10 samples in the validation set are chosen from the leaving samples; and testing set consists of all the remaining samples.
In the experiments, two classical approaches, EPO [21] and OSC [15], are adopted for comparison to purposefully illustrate the validity of the proposed MIRNet. A spectral quantization method using support vector regression (SVR) is used as a contrast test to explore the dependence of different methods on the training sample size. At the same time, three deep-learningbased methods, DNN, traditional CNN (CNN), and denoising CNN (DnCNN) [52], are designed to verify the performance of MIRNet. Specifically, the compared EPO method has one important parameter named the number of EPO dimensions c. The optimal threshold value of c is designed to be between 1 and 6 and is chosen to be 2. The compared OSC method has one important parameter named the number of filter factors k. The optimal threshold value of k is also designed to be between 1 and 6 and is chosen to be 4. The hyperparameters of the SVR are set through cross validation. The kernel function is radial basis function and its coefficient is set to 0.0001. The penalty factor for the wrong term c is set to 100 and epsilon is set to 0.01.
For deep-learning-based methods, the SEM in MIRNet has two 1-D ghost modules. Each module contains 32 and 16 convolutional kernels of size 5 and is followed by a maximum pooling layer of size 5. The CEM in MIRNet used in this study has two convolutional layers. Each layer contains 64 and 32 convolutional kernels of size 5 and is followed by a maximum pooling layer of size 5. MIRNet is optimized with Adam and a learning rate of 0.0001 by minimizing the D-S loss, and the batch size is 64. The training set and validation set in S 1 are used to determine the parameters and hyperparameters of MIRNet in the training process. The epoch is set as 3000, and the parameters received in the last epoch are used in the testing set in S 2 . For a fair comparison, the hyperparameter settings of DNN, CNN, and DnCNN are the same as MIRNet. The experimental results are reported by averaging the outputs of 20 independent runs.
The CNNE used to estimate SOC content in this study has two convolutional layers. Each layer contains 64 and 32 convolutional kernels of size 5 and is followed by a maximum pooling layer of size 5. The CNNE model's initial learning rate is set to 0.001. The number of training epochs is set as 1000, using a batch size of 64 and the Adam optimizer.
In this study, the distance between the spectral vectors is used to evaluate the accuracy of SMC influence removal. The distance d is measured by the Euclidean distance d(D i ,D i ) as follows: To compare the distance between the spectra before and after EPO and OSC processing with the deep learning-based methods, the correlation coefficient r is introduced to judge the similarity. The normal value range of r is [0, 1]. The closer it is to 1, the higher the similarity is. In this study, the determination coefficient R 2 score (R 2 ) and root mean squared error (RMSE) were used to evaluate the estimation accuracy. The normal value range of R 2 is [0, 1]. The closer it is to 1, the stronger the estimation ability of the model is. The specific calculation formula of R 2 and RMSE is shown as follows: where y i is the measured SOC content,ȳ i is the average of the measured SOC content, andŷ i is the estimated SOC content.

B. Parameter Analysis
A key parameter dilated rate in the methods using DiCNN is to affect the level of feature extraction by enlarging or decreasing the size of the receptive field. Thus, the accuracy of the SMC influence removal can be affected for the reason. To verify the influence of the dilation rate, different values of the dilation rate are analyzed on four DiCNN-based methods with 10, 50, and 90 percent of samples in development set S 1 . These four methods include a method using the CEM, a method combining CEM and Residual Net (CEM+Res), a method combining SEM and CEM (SEM+CEM), and the proposed MIRNet. The dilation rate is an integer greater than 0. When the dilated rate is equal to 1, the dilated convolution is equivalent to the ordinary convolution. Therefore, the influence of the dilation rate between 2 and 6 on the results is discussed in this experiment. Distance and R 2 are used to quantify the effects on the performance of the SMC influence removal methods.
The related experimental results are shown in Figs. 5 and 6. As shown in Fig. 5, when the dilation rate is 3, the distance between the spectra before and after SMC influence removal is the smallest in all these methods when using different scales of training samples. This also represents the least difference between the spectra. As the dilation rate continues to increase, the distance becomes increasingly larger. This is because when the dilation rate is too large, the size of the receptive field exceeds the range of the spectral band with greater correlation, leading to the introduction of meaningless information. Similarly, it can be seen from Fig. 6 that when the dilation rate is 3,  the accuracy of SOC content estimation using the spectra after removal is also the highest. R 2 decreases with the increasing dilation rate. Therefore, the value of the dilation rate is set to 3 in all the DiCNN-based methods to obtain the most valuable context information in this experiment. Accordingly, the sliding coefficient K in the loss function is set to 3.

C. Ratio of Training Samples Analysis
To further evaluate the generalization performance of the proposed MIRNet, Fig. 7 illustrates the evolution of distance and R 2 when the ratio of training samples in S 1 changes. In Fig. 7(a), the variation of the performance in the proposed MIRNet and three comparison deep-learning-based methods, SVR, CNN, DNN, and DnCNN, is analyzed. In Fig. 7(b), EPO and OSC are added for comparison. The ratio of training samples ranged from 10 to 90.
As expected, the distance decreases, and R 2 increases as the ratio of training data increases. The DNN method achieves the best performance using 10% of training samples. However, as the ratio increases, the performance of the DNN does not show a significant improvement. This trend may be caused by the small number of parameters to be fitted for the DNN. Similarly, the accuracy of SVR is better than that of CNN and DNN on all numbers of samples, which indicates that the method is less dependent on the number of samples and has a good fitting ability. In contrast, CNN, DnCNN, and MIRNet need more training samples to achieve better results due to the large number of parameters to be fitted. Therefore, as the scale increases, a significant performance improvement is shown. As seen from the experimental results, except at a ratio of 0.1, the proposed MIRNet obtains the lowest distance and highest R 2 compared to the other methods. This observation demonstrates that MIRNet has a better generalization performance than the other methods.
Examining the rangeability of distance and R 2 as the ratio increases, although the overall trend indicates an improvement in performance, there are still subtle differences. The distance shows a significant decrease with the increase in the ratio on all these approaches. However, the rise of R 2 decreases after the ratio reaches 0.5. This may be because although the spectra after removing the SMC influence are very close to the corresponding dry soil spectra, there is still noise that affects SOC content estimation in these spectra. Moreover, when the number of training samples is increased to a certain extent, the remaining

D. Ablation Study
To validate the effectiveness of each component (i.e., SEM, CEM, and Res) in the proposed method MIRNet, an ablation study is performed. The mean distance, r, R 2 , and RMSE on testing set S 2 of different combinations of these modules when using 90% of samples in S 1 are shown in Table II .
From the comparisons of SEM and CEM, it can be found that the methods using CEM (CEM and CEM+Res) improve the performance of SMC influence removal by a certain extent compared with the methods using SEM (SEM and SEM+Res). For example, the values of R 2 increase from 0.892 of SEM for 0.913 of CEM, and from 0.918 for SEM+Res to 0.921 for CEM+Res. It can be proven that the dilated convolutional layer in CEM can effectively extract the context information between the spectra. This contextual information has high guiding significance for the removal of the influence of SMC on soil spectra.
In addition, combining both SEM and CEM is better than one component (SEM or CEM) in terms of distance, r, R 2 , and RMSE. This is because by using SEM and CEM simultaneously, spectral features and contextual characteristics can be obtained at the same time. In this way, the influence of SMC in moist soil spectra can be fully explored to obtain a better removal effect.
From the comparisons on whether to use Res, it is found that the methods using Res (SEM+Res, CEM+Res, and SEM+CEM+Res) achieve a better performance of SMC influence removal than the methods without using Res (SEM, CEM, and SEM+CEM). For example, distance achieves decreases of 25.51%, 0.95%, and 46.04% on these three pairs of methods. Therefore, it is more effective to learn the influence of SMC through wet soil spectra than to directly fit the corresponding dry soil spectra.
Based on the conclusions of the aforementioned comparative experiments, SEM, CEM, and Res in the proposed MIRNet are found to be effective components for the removal of the SMC influence. They work collaboratively to render the confidence level of SMC influence identification and satisfactory removal performance of the deep-learning-based methods.

E. Spectra Comparison After Removing the SMC Influence
To display the effect of the SMC influence removal by each method more intuitively, this part draws the soil reflectance spectral curve after SMC influence removal at different SMC levels. The removal results of CNN, DNN, DnCNN, and MIRNet are visually shown in Fig. 8. The removal results shown in Fig. 8 are obtained using 90% of the samples in S 1 . All the spectral reflectance curves are averaged on all the samples in the S 2 set. It is obvious that these deep-learning-based methods effectively eliminate the influence of SMC on the soil spectra. The spectral reflectance curves after removal are very close to those of dry soil (SMC = 0). From the whole reflectance spectral curves, the moist soil spectra after removal by MIRNet are closer to the dry soil spectra than other approaches. It can be seen from the spectral reflectance curves that when the SMC is small (SMC ≤ 16), the removal effect is better.
To show the performance of all these methods more intuitively, the spectral reflectance curves of two spectral bands more subjected to soil moisture are amplified in these figures.
These bands are 1350-1550 nm and 2110-2310 nm. In these ranges of the spectral band, the moist soil spectra treated by SVR, CNN, and DnCNN have great differences from the dry soil spectra compared with MIRNet. For the DNN, the treated spectra are still much lower than the dry soil spectra at almost all SMC levels. In general, the variation in the average spectral reflectance curves of each SMC gradient in the S 2 set processed by deep-learning-based methods is relatively consistent. This demonstrates obvious similarity, and the difference between these spectral reflectance curves is small. This shows that these methods can effectively reduce the influence of SMC on soil

F. SMC Influence Removal Results of all the Proposed Methods
To better verify the performance of the proposed MIRNet, the SMC influence removal results of MIRNet and other comparison methods are sorted out in detail in Table III. The evaluation indices listed in the table include the distance and r between spectra before and after SMC influence removal and the accuracy of SOC content estimation by using the spectra before and after removal on testing set S 2 . Each removal method is constructed using 90% of the samples in S 1 . Since the spectral data processed by the spectral conversion methods PDS and OSC are no longer comparable with the original data, the spectral distances of these two methods are not listed in Table III.
The correlation coefficient r of the dry and moist soil spectra before SMC influence removal is 0.829. This suggests that there is some correlation between these spectra. The detailed SOC content estimation results of soil spectra before SMC influence removal are presented in the first column in Table III. The average testing R 2 and RMSE on the dry soil spectra in S 2 are 0.96 and 0.22, respectively. However, the average testing results on the original moist soil spectra are far apart from them. The most accurate testing R 2 and RMSE, using spectra with 4% SMC, are only 0.15 and 1.03, respectively. From the results, one can see that because of the influence of SMC, it is very inaccurate to estimate SOC using moist soil spectra. In other words, if the in-field spectra are directly input into the SOC content estimation model based on lab-dry spectra, the prediction accuracy will be very low. This also means that to establish the relationship between the in-field spectra and the existing large laboratory-dry soil spectral database, the influence of SMC in the in-field spectra must be removed first.
The results of the proposed MIRNet are presented in the last column in Table III. From the results, one can see that the proposed MIRNet exhibits the lowest distance and the highest estimation accuracies on all the spectra in the S 2 set. MIR-Net decreases the distance with SVR and other deep-learningbased methods, CNN, DNN, and DnCNN, by 60.82%, 65.99%, 70.49%, and 61.08%, respectively. It increases the R 2 with SVR, CNN, DNN, and DnCNN, by 2.05%, 3.67%, 6.48%, and 2.05%, respectively. EPO and OSC obtain low performance compared to the other deep-learning-based methods, and their mean R 2 values are only 0.703 and 0.747, respectively. It is obvious that these two methods do not obtain a reliable SMC effect through a spectral transformation in complex moist soil spectra. Compared with CNN and DNN, the quantitative removal method SVR shows better accuracy. This indicates that SVR has a good removal ability on the dataset with small training sample size. However, this method still has deficiencies in complex feature extraction, so its accuracy is still unable to be compared with MIRNet.
To analyze the performance of each method in a more detailed way, the removal results of the SMC influence of each method on the moist soil spectra with different levels of SMC are listed in detail in Table III. According to the data, with the increase in the SMC, the estimation accuracy of moist soil spectra before and after treatment shows a decreasing trend. This is because the high SMC introduces more complex effects, which greatly reduces the performance of both the influence removal model and the SOC content estimation model. The proposed MIRNet achieves the highest estimation accuracy in all SMC grades except 28% SMC. From these experimental results, one can see that the proposed MIRNet has a strong ability to solve the SMC influence removal problem by making full use of the powerful feature extraction ability of the CNN and the idea of Res.
To show the SOC content estimation using spectra after SMC influence removal of each of the proposed MIRNet and other comparison methods more intuitively, Fig. 9 shows the contrast between the measured and predicted SOC values of the testing samples in the S 2 set. It is obvious that the compared methods EPO and OSC exhibit very ordinary estimation accuracy [see

G. SMC Influence Removal Results of Different Soil Types
In the aforementioned experiment, yellow-brown soil and black soil samples are considered as a whole. In order to analyze the effect of the SMC influence removal methods in this study on the samples of different soil types, the two types of soil are further trained separately. The sets S 0 , S 1 , and S 2 of each soil type contains 25, 50, and 25 samples, respectively. Similarly, 90% samples in S 1 are used as training samples to train the SMC influence removal methods. To better verify the performance of the methods on different soil types, the SMC influence removal results of MIRNet and other comparison methods on the two soil types are sorted out in detail in Table IV. It can be seen from the experimental results that the removal accuracy of the two soil types modeled separately still shows the same results in different methods. The R 2 of brown-yellow soil and black soil using MIRNet are 0.914 and 0.905, respectively, and are higher than other methods. Simultaneously, one can see from the results that the modeling accuracy of yellow-brown soil samples is slightly higher than that of black soil in all removal methods. This phenomenon may be caused by the fact that the distribution difference of SOC in yellow-brown soil in this dataset is smaller than that in black soil.

H. Running Time
The experiments are run on a computer with an Intel Core i7-6200 U processor with a 2.30-GHz CPU and a GeForce GTX 970 graphical processing unit. Table V reports the total number of parameters and training time of different approaches when using 90% of samples in S 1 .
Compared to the reference methods, the proposed MIRNet has a similar number of parameters because the structure of the ghost module effectively reduces the number of parameters that need to be fitted. For the DnCNN method, the number of parameters is identical to the ordinary CNN. This shows that Res does not introduce any more parameters. For the DNN method, the training time is faster in comparison to other deep-learningbased methods. Specifically, the training of DNN only takes 160.35 s, but the performance is not optimal. Similarly, PDS and OSC also have this problem, the training speed is fast, but with poor results.

V. DISCUSSION
To solve the problem that soil spectra collected in situ will be affected by SMC and that the estimation accuracy is very low when such spectra are directly used to estimate SOC content, this study proposed an SMC influence removal model based on deep learning. The proposed MIRNet directly fits the influence of SMC on soil spectra through the idea of Res, which improves the removal accuracy. Moreover, the soil spectra removed by MIRNet are closer to the corresponding dry soil spectra through the proposed D-S loss.
The results of the aforementioned experiments effectively prove the advantages of the proposed MIRNet from multiple perspectives. First, MIRNet successfully removes the effects of SMC on soil reflection spectra. It can be seen from Table III that the removal effect of this model is not only better than the traditional EPO [21] and OSC [15] but also better than the relatively advanced SVR quantitative method and multiple deep learning models. It is particularly worth mentioning that the processing effect of this model is better than that of the DnCNN [52] method, which is widely used in other data denoising fields. Second, compared with the EPO and OSC methods, MIRNet can directly obtain reconstructed dry soil spectra. In addition, Table III and Fig. 9 show that the EPO and OSC methods have poor effects on the removal of the spectra with high SMC, while the deep learning method and MIRNet have better effects on these parts. This is because quantitative methods such as SVR and deep learning have successfully learned the relationship between the change in the spectra and SMC. Third, it can be seen from Fig. 8 that compared with other methods, the shape of the soil spectrum treated by MIRNet is the closest to that of the corresponding dry soil spectrum. This is because by using D-S loss in the training, the spectral shape is also used as an important index in MIRNet to evaluate the model effect.
The experimental data in this study are all spectral data collected in the laboratory. However, the important requirements of smart agriculture at present are rapid and accurate SOC estimation and large-scale soil investigation. With the rapid development of remote sensing technology, many multimodal data with complex and heterogeneous observations can be obtained. If these remote sensing data are combined with interpretive data such as SMC and roughness in large areas, they can form an excellent data source for large-scale soil investigation. Deep learning has been successfully applied to multimodal remote sensing data processing due to its ability to mine deep features and powerful processing capabilities. Therefore, the proposed MIRNet provides a theoretical basis for such large-scale soil investigations.
Admittedly, the main limitation of this study is the relatively small size of labeled soil samples, as deep learning methods are highly dependent on the training sample size. Therefore, when designing the structure of MIRNet, we built a lightweight network by controlling the number of layers and modifying the structure. It can be seen from Fig. 7 that when the proportion of training samples increases to 50%, the distance still tends to rise, but the range of accuracy improvement is obviously reduced. This indicates that the current training sample size is sufficient for the proposed MIRNet. However, such a lightweight network may have limitations if there are requirements for SMC influence removal in a large range of soil data. If a network with a more complex structure is to be trained, the corresponding training sample size also needs to be expanded. Unfortunately, due to the complexity of soil sample preparation and the expensive cost of soil SOC chemical testing, it is difficult to expand the sample size. Therefore, in future studies, we hope to expand the number of training samples to try to construct a deep-learning-based SMC influence removal model that can meet more complex tasks.

VI. CONCLUSION
In this study, the idea of removing SMC influence with a deep-learning-based method was investigated for the first time. MIRNet with three important modules (SEM, CEM, and Res) was proposed for SMC influence removal. First, SEM explored spectral feature extraction by building a ghost module, which provides an excellent SMC influenced characteristic screening ability and effectively addressed the problem of a limited number of training samples. Second, CEM explored spectral context information extraction by building DCNN, which took advantage of correlations between spectral bands. Third, Res is used to learn the SMC influence from moist soil spectra rather than directly fitting dry soil spectra. Finally, MIRNet is used to build a trustworthy system for SMC influence removal in the combination of these three modules.
The experimental results showed that the proposed MIRNet improved the performance of SMC influence removal and SOC estimation with the processed spectra at the same time. Using the trained removal model, the effect of SMC on the spectra can be effectively removed under the condition of unknown SMC. This study provides a theoretical reference for the rapid monitoring of soil fertility information under the condition of unknown SMC in the field with deep-learning-based methods. In practical application scenarios, the proposed MIRNet can not only be used for soil property content estimation based on spectra data collected in the field but also be extended to large-scale soil properties content monitoring based on hyperspectral remote sensing data of aviation or astronautics. At the same time, this method can also be used to remove the influence of other soil characteristics in soil spectra, which is an important research direction in future work. His current research interests include plant diagnosis, image processing, and deep learning.