A Surrogate Model Based on Artificial Neural Networks for Wave Propagation in Uncertain Media

Soil materials can exhibit strongly dispersive properties in the operating frequency range of a physical system, and the uncertain parameters of the dispersive materials introduce uncertainties in the simulation result of propagating waves. It is essential to quantify the uncertainty in the simulation result when the acceptability of these calculation results is considered. To avoid performing thousands of full-wave simulations, an efficient surrogate model based on artificial neural networks (ANNs) is proposed in this paper, to imitate the concerned ground penetrating radar (GPR) calculation. With the autoencoder neural network to reduce the dimensionality of data, the surrogate model successfully predicts the outputs of the GPR calculation using a small number of training samples. The finite-difference time-domain method with the uniaxial perfectly matched layer is used to collect sampling data for the surrogate model. The process of constructing the surrogate model is presented in detail in this paper. The proposed surrogate model is demonstrated to be an attractive alternative to the full-wave GPR calculation due to its considerable advantage in terms of computational expense and speed.


I. INTRODUCTION
The numerical simulation is an alternative interpretation of wave propagation in ground penetrating radars (GPRs), and it relies on a set of input parameters which can affect the electromagnetic pulses and then the survey of a target consequently [1]. GPRs are important remote sensing tools in many fields such as civil engineering [2], landmine detection [3], and environmental applications [4]. It is of great importance for the study of numerical modeling of GPR systems. A lot of numerical methods have been employed for GPR system modeling [1]. Among these numerical methods, the finitedifference time-domain (FDTD) method [5] is one of the commonly used methods because it is easy to implement and it can also model dispersive and lossy media [6]. In practice, the exact values of the inputs are always unknown, leading to the uncertainties in the output of the simulation [7]. Quantifying the uncertainty in the simulation result is an indispensable The associate editor coordinating the review of this manuscript and approving it for publication was Sunil Karamchandani . part in GPR calculation when the acceptability of the output is considered [8].
The methods used to quantify uncertainty in simulation results can be divided into two categories: non-intrusive methods and intrusive methods [7]. The traditional nonintrusive method is Monte Carlo simulation (MCS) which requires running the deterministic simulation code several thousand times to converge, resulting in a high computational cost [7]. In [8], the authors propose an intrusive method which implements generalized polynomial chaos expansion (gPCE) into the auxiliary differential equation (ADE) FDTD [6] to quantify uncertainty induced by uncertain parameters. A considerable computational advantage over MCS is achieved. However, one of its limitations is that the computational complexity increases rapidly with the increasing number of uncertain input parameters [8]. To solve this problem, it is essential to identify important uncertain parameters from all inputs for some applications, which is not easy to achieve in practice. Moreover, the intrusive gPCE is efficient for a relatively small degree of random perturbation in the inputs of the simulation, however, for most of the practical engineering problems, the variation in uncertain inputs is greater or equal to 10% which is difficult for intrusive gPCE to handle [9].
To overcome the above problems, a surrogate model based on artificial neural networks (ANNs) [10], which can be computed very efficiently, is constructed to mimic the behavior of the GPR simulation model. ANN is a brain-inspired system which aims to imitate how humans learn. Various advanced neuralnetwork structures have been investigated for an input-output relationship [11]. The sampling of propagating waves in FDTD usually results in high-dimensional data. However, a simple input-output ANN has difficulty in handling a relatively small number of high-dimensional training samples because the insufficient training samples lead to inaccurate results. Therefore, effective feature learning methods are critically needed to automatically capture the useful features of the high-dimensional data. For the reasons mentioned above, an autoencoder neural network [12]- [14] is pre-trained and introduced into the proposed surrogate model to map the high-dimensional outputs to a suitable lowdimensional space, and also for reconstructing the original high-dimensional data. It can be divided into two separate networks: an encoder and a decoder [12]. The input dataset for the training of the surrogate model consists of two parts: the uncertain parameters of soil and the encoder output that converts high-dimensional electric field data from FDTD to a low-dimensional code. In the testing process, the trained surrogate model maps the relationship between the uncertain parameters and the low-dimensional code, and the decoder network then recovers the electric field data from the code. The statistical quantities of a GPR system from the surrogate model, such as the mean and the standard deviation, are provided for comparison with the results from MCS. The calculation results verify the accuracy and efficiency of the proposed model. This paper is organized as follows. The FDTD simulation method and the proposed surrogate ANN model for UA are provided in Section II. Section III gives the description of a numerical example of the proposed model. Section IV draws the conclusion.

II. PROPOSED SURROGATE MODEL FOR GPRs CALCULATION A. ADE-FDTD SIMULATION OF GPR SYSTEM
In this paper, a two-dimensional (2-D) GPR system is presented, and the uniaxial perfectly matched layer (UPML) [6] is used as the absorbing boundary condition (ABC). Maxwell's equations for a wave propagating in 2-D are defined as: where H x represents the magnetic field oriented in the x-direction, H y represents the magnetic field oriented in the y-direction, and E z represents the electric field oriented in the z-direction. µ is the permeability, and is the permittivity. The solutions to Maxwell's equations can be found by using the FDTD method.
In this paper, the GPR system model has soil and solid metallic target. The soil is considered as a nonmagnetic medium with frequency-dependent dielectric permittivity, and it is modeled by a two-term Debye model with a static conductivity σ s [1]. The parameters of Debye model are presented in Table 1, which are obtained by measurement [1]. They have uncertainties due to measuring tools, manual operation or other reasons. In practice, the exact values of the input parameters are unknown. Assuming there are seven uncertain input parameters of ∞ (θ ), s (θ ), A p (θ ), τ p (θ ) (p = 1, 2) and σ s (θ ) in the complex relative permittivity r (ω, θ), the form of r (ω, θ) is where ω is the angular frequency, θ is a random variable, and j 2 = −1. For the analysis of 2-D propagating waves using ADE-FDTD, the first auxiliary variable for the electric field E z is where W h are associated with the x, y and z normal planes, respectively, and the form of W h is [6] The details of s h and σ h are presented in [6]. The second auxiliary variable is The third auxiliary variable is Submitting (5), (7) and (8) into the update equations of ADE-FDTD, we obtain the electric field E z and the magnetic fields H x and H y . In the iteration provided in (9), (10), (11), and (12), the three auxiliary variables are written as L k z (nx, ny, θ), D k z (nx, ny, θ) and R k pz (nx, ny, θ), where nx and ny are space steps along the x-and y-directions, respectively, and k is the time step. x and y are sampling widths along the x and y directions, respectively, and t is the time interval. The uncertain parameters of the soil model produce uncertainties in simulation results. The electric field E z and the magnetic fields H x and H y are written as E k z (nx, ny, θ), H k x (nx, ny, θ), and H k y (nx, ny, θ) in the iteration process. The uncertainties in the calculation results require to be quantified.

B. PROPOSED SURROGATE MODEL BASED ON ANNs
The training and testing processes of the new surrogate model for GPRs calculation are shown in Fig. 1. The goal of the proposed surrogate model is to predict the outputs in GPRs when a set of uncertain parameters of ∞ , s , A p , τ p , and σ s is input. To handle high-dimensional data from ADE-FDTD outputs, such as E z observed at a certain location in all time steps, the autoencoder neural network is introduced into the proposed surrogate model to map the high-dimensional outputs to a suitable low-dimensional space.
where   The encoder and decoder used in the proposed surrogate model are from an autoencoder neural network which needs to be trained before constructing the proposed surrogate model. The traning process of an autoencoder neural network is presented in Fig. 2. The encoder maps the input data from a high-dimensional space into codes in a lowdimensional space, and the decoder reconstructs the input data from the corresponding codes. Given the training data represents a D-dimensional vector), the encoder transforms the input matrix of U into a hidden representation of C = {C 1 , C 2 , C 3 , . . . , C L }(C l ∈ R d ) through activation functions, where d D, and L is the number of training samples of the autoencoder neural network. Then, the matrix of C is transformed back to a reconstruction matrix of U = {U 1 , U 2 , U 3 , . . . , U L } by the decoder. The input data U are the electric fields or the magnetic fields calculated from ADE-FDTD, and D is the number of time steps.
In the process of dataset generation, the sampling method to obtain input parameters of ADE-FDTD is latin hypercube sampling (LHS) [15]. It is a sampling method enabling to better cover the domain of variations of the input variables, thanks to a stratified sampling strategy. The sampling is undertaken as follows [15]: 1) the range of each input variable is stratified into isoprobabilistic cells; 2) a cell is uniformly chosen among all the available cells; 3) the random number is obtained by inverting the cumulative density function locally in the chosen cell; 4) all the cells having a common strate with the previous cell are put apart from the list of available cells.
Normalization is applied to pre-process data. The dataset of each ANN is split into three parts: a training set, a validation set, and a test set. The training data represent 60% of the whole data. The number of the training data of the autoencoder neural network and the proposed surrogate model are 100 and 200, respectively. The parameters are optimized through adaptive moment estimation (Adam) [16], [17]. The learning rate is 0.001. The rectified linear unit (ReLU) function [18], [19] is used as activation functions both in the hidden layers of the autoencoder neural network and the proposed surrogate model. The ReLU function is where a i is the input to the ith neuron. The linear activation function [16] is chosen as the activation functions in the input layer and the output layer of the ANNs. The mean squared error (MSE) [19] is used for performance evaluation in the ANNs where Y r andŶ r denote the observed and forecasted values, respectively, of the rth datum, and R is the total number of the data. The validation data represent 20% of the whole data, which are used to provide an unbiased evaluation of a model fitting on the training data while the model hyperparameters such as number of hidden layers and units, activation function, learning rate, etc. are tuned. The number of the validation data of the autoencoder neural network and the proposed surrogate model are 34 and 67, respectively. The test data represent 20% of the whole data, which are used to provide an unbiased evaluation of a final model fitting on the training data. The number of the test data is same as the validation data.
In the training process, overfitting is the most troublesome issue. The overfitting model doesn't generalize well from the training data to new data. To reduce overfitting, the dropout method [20], [21] is applied to the training process of the proposed surrogate model and the autoencoder neural network. The idea of dropout is to randomly drop neurons from the neural network during training and it is similar to sampling a sub-network from a larger network [20]. The idea of dropout is presented as follows [20] where a g is the input vector of layer g, y g is the output vector from layer g,ỹ g is the thinned output vector of layer g, w g are the weights, b g are biases, * denotes an element-wise product, and f is the activation function. For any layer g, r g is a vector of independent Bernoulli random variables each of which has probability q of being 1. In the testing process, the weights are scaled as w g test = qw g , which is similar to averaging the predictions produced by a large number of different networks but in a reasonable time [17]. In Fig. 3, the training loss and the validation loss of the proposed surrogate model with and without dropout are presented, respectively, while the other hyperparameters remain the same. The dropout method is applied to each hidden layer with the same dropout probability of q = 0.4. The traning error, validation error and the testing error of the surrogate model with and without dropout are presented in Table 2 (Epochs=1000). Different from the strategy used in the surrogate model, the dropout method is only applied to the two hidden layers adjacent to the code layer in the autoencoder neural network with the same dropout probability of q = 0.2. The dropout method applied to the two hidden layers has a better performance than that applied to each hidden layer in its architecture.
The training loss and the validation loss of the autoencoder neural network with and without dropout are presented in Fig. 4. The traning error, validation error and the testing error of the autoencoder neural network with and without dropout are presented in Table 3 (Epochs=5000). These results show that the dropout method can significantly reduce overfitting in the training process. It needs to be emphasized that for the proposed surrogate models trained with and with-   out the dropout method, the autoencoder neural network used in both cases is same and trained with the dropout method.   receiver, respectively. The position of Tx is (0.80m, 3.05m), and the position of Rx is (2.90m, 3.05m). The excitation is Blackmann-Harris pulse source as [1]

III. APPLICATION EXAMPLE AND RESULTS
where the center frequency f c = 200 MHz, and T s = 1.55/f c . Other parameters of the excitation are given in [1]. The center of the solid metallic medium is (1.80m, 0.80m) and its size is 0.30 m × 0.30 m. The metallic medium is buried 2.05 m below ground level. The details of the model are given in [8].
Compared with the intrusive gPCE method proposed in [8], the number of uncertain parameters in this paper is increased from one to seven, and the variation in each parameter is increased from 5% to 10%. When there are seven uncertain parameters in the complex relative permittivity r (ω, θ), the FDTD output E z observed in the position of Rx is presented in Fig. 6. It is shown that the output uncertainty with a variation of 10% in each uncertain input is much larger than that with a variation of 5% in each uncertain input. The intrusive gPCE has difficulty to handle the large uncertainty in output. The quantitative level of confidence held in the ADE-FDTD simulation results of GPRs is essential.

B. TRAINING THE SURROGATE MODEL
There are seven uncertain parameters in ADE-FDTD simulations and the variation in each uncertain parameter is 10%. The inputs and outputs in the FDTD simulation are both normalized before the surrogate model is trained. The dimensions of FDTD output and its code are 5000 and 200, respectively. It is required to run 100 FDTD simulations to collect the training samples of the autoencoder neural network, and the proposed surrogate model requires 200 training samples. The details of the hyperparameters of the proposed surrogate model and the autoencoder neural network are presented in Table 4.

C. QUANTIFYING UNCERTAINTY IN GPR CALCULATION RESULTS USING THE TRAINED SURROGATE MODEL
Once the surrogate model is trained, it can be used for various analyses such as quantifying the uncertainty in the simulation results. When there are seven uncertain parameters in ADE-FDTD simulations and the variation in each uncertain parameter is 10%, the mean and standard deviation evaluated with the surrogate model are presented in Figure 7. The    Table 5. Compared with running a thousand FDTD simulations in MCS, the proposed surrogate model largely reduces the number of FDTD simulations and improves the efficiency.
In this paper, all calculations are performed on an Intel i5-6440HQ 2.6GHz machine with 16GB RAM.

IV. CONCLUSION
This work aims to construct a surrogate model which mimics the behavior of a 2-D GPR simulation model as closely as possible. Subsequently, it is used to quantify uncertainty induced by uncertain parameters of the dispersive and lossy soil because the surrogate model can predict the outputs of the GPR simulation fast. The soil is modeled by a two-term Debye model with a statistic conductivity, and the ADE-FDTD method with the UPML as the ABC is used for calculation. The proposed surrogate model can quantify uncertainty in GPR simulation results with the large variation in each input parameter of the calculation. In addition, it does not require prior knowledge of the inputs such as the distributions of the random input variables. The results of the proposed surrogate model are consistent well with the results of MCS which performs a thousand of full-wave simulations. The new method also shows a much faster calculation speed. To obtain training samples of the proposed surrogate model, multiple full-wave simulations are required, which is time-consuming. The future work will focus on reducing the number of samples required for the training process in the surrogate model with high accuracy.