Enhanced PML Based on the Long Short Term Memory Network for the FDTD Method

This paper proposes a new absorbing boundary condition (ABC) computation approach based on the deep learning technique. Benefited from the sequence dependence feature, the Long Short-Term Memory (LSTM) network is employed to replace the conventional perfectly matched layer (PML) ABC for the Finite-Difference Time-Domain (FDTD) solving process. The newly proposed LSTM based PML model is trained by the electromagnetic field data on the interface of the conventional PML. Different from the conventional PML, the newly proposed model only needs one cell layer as the boundary. Hence, the newly proposed method conveniently reduces both the algorithm’s complexity and the area of computation domain of FDTD. Additionally, the newly proposed LSTM based PML model can achieve higher accuracy than the conventional artificial neural network (ANN) based PML, thanks to the sequence dependence feature of the LSTM networks. Numerical examples have illustrated the capability and the accuracy of the proposed LSTM model. The results illustrate that the new method can be compatibly embedded into the FDTD solving process with the high accuracy.


I. INTRODUCTION
Absorbing boundary conditions (ABCs) are widely utilized to truncate the computational area for unbounded problems during FDTD solving process [1]- [3]. Perfectly matched layer (PML), as the most widely-used ABC, has been developed in various application scenarios, thanks to its high tolerance to frequency and its excellent absorption capability over a broad scope of angles [4], [5]. Also, PML has been widely reported in various application of electromagnetic modeling, including convolutional PML (CPML) [6] and uniaxial PML (UPML) [7], [8]. But, the PML size has to be augmented to decrease the residue error for better absorption performance over wide range angles and broad frequency band. Hence, the expanded size leads to the increased computational complexity and computational domain.
Accompanied with the continuous development of computer science, machine learning (ML) approaches [9] have numerous successful application scenarios, including electromagnetic computation [10]- [13], electromagnetic inverse are: (1) The LSTM based PML model reduces the computation domain and thereby the computation complexity for the FDTD ABC, because this new model only needs one-layer cell; (2) The newly proposed LSTM based PML is very convenient and flexible to be implemented in various FDTD application scenarios; (3) The accuracy of this proposed LSTM based PML model has been greatly increased (about 5dB), compared to the previous ANN based PML [20].
The structure of this paper is: the formulation of FDTD and the newly proposed LSTM model are described in Section II. Meanwhile, Section II also demonstrates the mechanism of integrating the proposed LSTM model to FDTD. Numerical benchmarks in the next section present the feasibility and accuracy of the proposed LSTM method, while the conclusion is offered in Section IV.

II. FORMULATIONS A. FORMULATION OF CONVENTIONAL PML
The conventional PML medium in a 2D Transverse Electronic to the z direction (TE z ) wave case is shown in Fig. 1(a), where the EM field in PML consists of four components (E x , E y , H zx , H zy ) as described by the Yee cell [2], [23], specifically demonstrated by the four following equations: where δ is the PML thickness, and ρ is the distance away from the interface. The conductivity in (1), as shown at the bottom of this page, is represented as σ (ρ) = σ max ( ρ δ ) p with p chosen as 1 or 2, while σ max acts as the conductivity at the outermost layer of PML.

B. MECHANISM OF LSTM BASED PML
The LSTM network is a representative deep recurrent neural network (DRNN), which can learn long-term dependencies between time steps of sequence data [24]- [29]. LSTM is designed to overcome the inherent problems of RNNs, such as vanishing and exploding gradients [24]- [30]. Besides, LSTM block replaces the hidden layer mode for traditional ''shallow'' ANN. The LSTM block in Fig.2(b) can ''store'', ''write'' and ''read'' data via gates that open and close [26]- [30], which makes it matching the nonlinearities of real sequence data better than traditional ANN and RNN [24]- [30]. Considering the computation process within conventional PML, each cell in conventional PML repeatively computes local field based on its local and neighbouring fields at the previous and current time step [2], [5], [23]. Therefore, this computing process could be referred as the nonlinear sequence-dependence computation and it can be handled by LSTM model. By opening and closing gates, LSTM cells can 'forget' the potential useless information in previous time steps, 'update' state and 'improve' the output for the current time step in FDTD solving process. The core component of an LSTM network is the LSTM layer, where long-term dependencies between time steps of (1) VOLUME 8, 2020 sequence data are learnt, as is presented in Fig. 2. The time series or sequence data x t are input into this network by the sequence input layer. In Fig.2, the flow of a time series x t of length S is input through the LSTM layer, where h t is known as the hidden state and c t denotes the cell state. The output y t is ultimately obtained by doing linear transformation to h t . At the time step t, the LSTM block makes use of the previous state c t−1 , h t−1 and the current input x t , and then updates h t and c t for the current time step. The hidden state and the cell state involves information learned from the previous time steps. Thus, the LSTM layer can absorb or delete information by using the cell state at each time step. These updates are totally controlled by gates of the LSTM layer, presented in Fig.2 The mechanism of LSTM is based on the following equations (2)- (8). The specific computation process within equations (2)-(8) is as follows: the initial state and the sequence in the first time step is used to calculate the first output and update the cell states. Then, for each time step t, the LSTM block conducts the same operation, utilizing the state and the input sequence to calculate the output and cell state. At the time step t, the Fig.2(b) illustrates how the gates forget, update, and output the hidden and the cell states in the LSTM block. The layer state includes the hidden state and the cell state. The important sequence information is involved in the cell state. For each time step t, the layer 'store' or 'forget' information from the cell state [24]- [30], as described in the equations (2)- (8).
The learnable weights of LSTM layer are output weight U, g t , f t and o t respectively represent the input gate, forget gate, cell candidate, and output gate at time step t, while denotes the Hadamard product. Plus, the state activation function σ c is denoted by the hyperbolic tangent function, and the gate activation function σ g is chosen as the sigmoid function [24]- [30].
The application of the proposed LSTM PML consists of two stages: the off-line training stage and the online prediction stage. In the offline training stage, the EM field data on the interface between the first layer of conventional PML and the object domain are used to train the deep LSTM network model, so that the prior information of FDTD solving process in the PML is integrated into the trained model. In the online stage, the trained LSTM PML is integrated into the FDTD solving process to do the further field prediction with satisfactory accuracy and efficiency.
The specific offline training process of the proposed LSTM model is presented in the next section. For the proposed LSTM model, the dimension of input and output are 12 and 3, respectively, to match the field information chosen as the training data. Based on the training dataset, the learnable weights W , R, U and b could be ultimately determined by various training algorithms. In this paper, Adaptive Moment Estimation (Adam) optimizer is chosen to optimize the halfmean-squared-error loss function, as reported in [31]- [35]. The training is done by 'sequence-to-sequence' method for LSTM [26]- [30]. Fig. 3 presents the mechanism of collecting data for training the LSTM based PML model. While the node P is on the 21030 VOLUME 8, 2020 first layer of conventional PML, the cells near P receive waves from different directions in Fig.3. Based on the wave in each direction, the local and neighbouring field data of the node P in the continuous time sequence are organized to form the training data. In Fig. 3, TE z wave is utilized as the example, where we set input data and output data as x t ij and y t ij : groups are collected on the interface of conventional PML with 40-layer thickness, which has been widely accepted to be sufficient to effectively absorb the electromagnetic waves and delete reflection between layer interfaces [2]- [5].
Several issues are still required to be discussed for creating our LSTM model, including number of hidden unit and sequence length of LSTM.

1) ACTIVATION FUNCTION
using the sigmoid and hyperbolic tangent activation functions makes gradient vanishing in conventional RNN, especially when the training algorithm Back-Propagation Through Time (BPTT) is utilized [24]- [30]. However, this problem is resolved by the network structure of a LSTM, thanks to the various gates and the memory cell [26]- [30].

2) SEQUENCE LENGTH
according to [27]- [33], the increase of sequence length S for training data set cannot only help the LSTM model improve accuracy, but also makes training converge faster. However, when increasing S up to a certain threshold, neither accuracy nor convergence can be improved any more [31]- [33].
In this case, more training computations are required due to the increase of S, which unavoidably degrades computation efficiency [27]- [33]. Ultimately, we set the sequence length as S = 15 for the proposed LSTM based PML model, which has been widely chosen in the numerous physical-based cases [31]- [33], [36]- [38].

3) NUMBER OF HIDDEN UNIT
in most cases, the determination of the exact number of hidden unit for h t is still an important research area for deep learning [18]- [20], [39]- [40]. In this paper, as the most commonly used strategy for deep learning, the trialand-error process is taken to ensure the number of hidden unit [18]- [20]. While various number of hidden units can be tried, the expected values are the smaller ones resulting into smaller training errors and less computation. Taking the precision and the number of parameters of the proposed model into account, we ultimately set the number of hidden unit as 20.
Therefore, the final LSTM based model is represented in the equations (2)- (8). This trained LSTM model with the prior information of FDTD solving process in the PML can then be integrated into the FDTD solving process to do the further field prediction with satisfactory accuracy and efficiency. Based on the resultant LSTM model, y t S ij can be predicted by x t S ij and the field information on the PML interface at t S+1 can be further obtained. Meanwhile, the states of each grid of one-cell LSTM based PML at t S+1 are computed and stored to be used in the next time step. The index i and j in this section represent that of the grid for the LSTM based PML, which act as the node P presented in Fig. 3. Then, using y t S ij , we can calculate field in the object domain at t S+1 and then gain x t S+1 ij , by standard FDTD process. The further predictions are done for boundaries at t S+2 , and then further time steps. By exactly repeating this process, the proposed LSTM model can replace the entire conventional PML and can be embedded to the FDTD solving process. Hence, for each grid on the LSTM PML, the field information can always be predicted by using field information on local and neighbouring grids. The program workflow is summarized as:

III. NUMERICAL EXAMPLES
We use several representative numerical benchmarks to validate the performance of this newly proposed LSTM based PML model. The conventional 1-cell PML, the conventional 5-cell PML and the HTBF based PML [20] are utilized to compare with the proposed LSTM based PML.

A. POINT SOURCE EXCITATION IN A SQUARE REGION
In the first benchmark, LSTM based PML is employed in 2D square area in TE z wave, presented in Fig. 5(a). While the chosen time step is t = 1ps, a 15mm×15mm square domain is discretized as x = y = 1mm cells. Fig. 5(b) shows the conventional PML scenario, where the same square area involves a 5-cell conventional PML at each boundary. 2) Train LSTM based PML model by using the data set in the above step, as is represented in (2)-(8).

Model Embedding
Initialization: 1) Initialize H t 0 and E t 0 in the object domain and one-cell LSTM PML as shown in Fig. 4. 2) Do zero-padding for c t and h t with t < t 0 for each cell on LSTM PML. kth Iteration:

3) Update H t k and E t k at one-cell LSTM based PML by using H t k−1 and E t k−1 in object domain, H t k−1 and E t k−1 in one-cell LSTM based PML, and c t k−1 and h t k−1 on each cell on LSTM PML 4) Update c t k and h t k for each cell on LSTM PML 5) Update H t k and E t k in object domain by using H t k and
E t k at one-cell LSTM based PML with standard FDTD process. But, the proposed LSTM based PML only needs one cell at each edge in Fig.5(a). At the position x 0 = y 0 = 10 mm, there is a sinusoidal magnetic current source in z direction, formulated as the following expression: where A = 1, f c = 10GHz, t 0 = 0. The Magnetic fields (H z ) at A (6,10) and B (2, 2) are probed. Normally, 100 time steps are seen as the steady state respond [1]- [3]. We use the following equation to define the relative error at each point: In this example, 5000 groups of training data is utilized to train our LSTM model in the offline training stage. We randomly choose 80% of the data groups to do training, 10% for validating, and another 10% for testing. The initial learning rate is 0.001, while the min-max normalization is adopted for better training [31]- [33], [41]. The proposed model is implemented in the Deep Learning Toolbox in Matlab 2019a [42]. The training of this LSTM model takes 34.83s CPU time. The performance of our LSTM model in the online application stage is demonstrated in Fig. 6, where the relative errors of LSTM PML model at Point A and B are compared both with the 5-cell and 1-cell conventional PML and with the previously proposed HTBF based PML [20]. As shown in Fig. 6(a), the maximum of relative error from LSTM based PML are about −30dB at Point A, while that maximum for 1-cell conventional PML, 5-cell conventional PML and HTBF based PML arrive at 0dB, −20dB and −25dB, respectively. Moreover, it can be seen that the maximum of relative errors at Point B for four methods are respectively 5dB for the 1-cell conventional PML, −10dB for the 5-cell conventional PML, −15dB for HTBF based PML and −20dB for LSTM based PML. Besides, Fig. 6(b) presents that the average of relative error is −20dB for both 5-cell conventional PML and HTBF based PML model, while it is below −25dB for our novel LSTM model. However, that average of 1-cell conventional PML is almost 0dB. From the error analysis above, we can conclude that 1-cell conventional PML cannot be offer any meaningful information for FDTD solving process, while the LSTM based PML provides the best accuracy for this case.
According to Fig.6, our LSTM based PML is evidently better than HTBF based PML and conventional PML. Besides, only one cell is used to finish absorbing for the LSTM based PML. Furthermore, CPU memory can be considerably saved by the proposed LSTM model because the FDTD computation process repeatedly uses the same model. The CPU time for the implementation of four methods are 0.347s for the 1-cell conventional PML, 0.486s for HTBF PML model, 0.624s for LSTM model and 0.715s for 5-cell conventional PML.

B. SCATTERING BY PEC OBJECT
While the second benchmark takes all similar setups with the Section III A, a PEC square with the size of 3mm×3mm is added into the 15mm×15mm square area.
In this benchmark, the same model trained in Section III A is employed to demonstrate the generalization of the proposed LSTM method. According to Fig. 8(a), the proposed LSTM based PML has better performance in its online application stage. The results present that the maximum of relative error of HTBF based PML and LSTM based PML and 5-cell conventional PML are all around −20dB at A point, while that relative error is nearly 0dB for the 1-cell conventional PML. Furthermore, the maximum of relative errors of HTBF based PML and 5-cell conventional PML are both around −10dB PML at Point B, while that of LSTM based PML is about −18dB. But, as the worst case, the maximum of relative error of 1-cell conventional PML can overcome 5dB at B point. Moreover, Fig. 8(b) presents the average of relative errors of the four methods. The average relative errors are about −20dB for HTBF based PML and 5-cell conventional PML, and 0dB for 1-cell conventional PML, while that average is below −25dB for the proposed LSTM model. The result of the error analysis above is the same as the previous section: 1-cell conventional PML cannot be used for FDTD, while the LSTM based PML provides the best accuracy.
From the comparison above, we can conclude that LSTM based PML is evidently better in terms of accuracy, while the accuracy of 5-cell conventional PML is approximated to that of the HTBF based PML. The CPU time for the implementation of four methods are 0.397s for the 1-cell conventional PML, 0.438s for HTBF PML model, 0.601s for LSTM PML model, and 0.833s for 5-cell conventional PML. VOLUME 8, 2020

IV. CONCLUSION
This work integrates deep learning approach into the absorbing boundary condition for the FDTD. The newly proposed LSTM based PML substitutes for the conventional PML ABC. Unlike the conventional PML, only a one-cell boundary layer is required by this newly proposed LSTM based PML model to absorb the propagating wave. Hence, it considerably reduces the size of the computation domain. Compared with the conventional PML and conventional ANN based PML, the new method provides both good accuracy and low computation complexity simultaneously. Numerical examples demonstrate that the newly proposed LSTM based PML model can successfully replace the conventional PML to be flexibly embedded to the FDTD solving process. The newly proposed method opens a new PML algorithm path to the deep learning regime. He also serves as a Reviewer for the IEEE TRANSACTIONS on several topics and other primary electromagnetics and microwave related journals.