Deep Learning-Based Prediction Algorithm on Atmospheric Turbulence-Induced Wavefront for Adaptive Optics

Correction performance of an adaptive optics (AO) system is severely limited by its system latency under high temporal frequency distortions. Wavefront prediction methods has been proven to be an effective way to compensate system delay. A novel wavefront prediction method based on residual learning fusion network (RLFNet) is proposed in this paper. The network is able to eliminate redundant information between adjacent wavefront frames and fuse refined features, which provides a higher prediction accuracy under fewer priori wavefront input. The method is tested on a 1km laser atmospheric transmission experimental setup through high frequency atmospheric turbulence, where wavefront data is collected from a 1kHz Shack-Hartmann wavefront sensor (SHWS). We show that the proposed architecture is able to predict the second frame wavefront after the previous 6 frames with a root-mean-square (RMS) wavefront error reduction up to 53% compare to non-predictive method.


I. INTRODUCTION
S UPPRESSING the degrading effect of atmospheric turbulence on laser beam transmission has been a major research focus in the fields of laser communications, and laser detection among others. AO technique is an effective way to compensate atmospheric turbulence in real time. However, real AO systems usually assume that the distorted wavefront is static during a control period, which leads to a servo-lag error of 2-3 sampling periods in the control loop [1], [2], [3]. The time delay in the loop mainly comes from signal processing and control tasks such as wavefront sensor data acquisition, wavefront reconstruction and control operation. Such time delay will cause the compensation wavefront on the deformable mirror (DM) lag behind the actual measured wavefront, which severely limits the correction performance of the AO system.
A large number of prediction methods have been studied to compensate the time delay. Among these prediction methods, the iterative prediction method like Recursive Least Squares (RLS) algorithm is to minimize the systematic error, which provides a simple calculation process and avoids to fall into the local minimum point. However, when the mode order of wavefront aberrations is too high, it cannot converge in a short time [4], [5]. Model-based prediction methods use atmospheric turbulence model to predict the change of wavefront, such as the Linear Quadratic Gaussian control algorithm, which has a good approximation performance, but it is difficult to balance the prediction accuracy of the model between the control complexity of the controller [6], [7], [8]. In addition, there are more complex Kalman Filter based methods [9], [10], [11], [12] and extensions of various approach [13], [14], [15], [16]. In fact, the majority of these algorithms can be described as analytical applications of various estimation frameworks. But amid a large wave of general advances in Neural-network (NN), NN-based methods have been widely used for wavefront reconstruction and prediction of AO systems [17], [18]. It turns out that NN-based methods have the advantage over analytical estimation techniques in terms of alleviating time delay, because NN predictors are self-optimizing, generalizable and have strong nonlinear fitting ability [18], [19]. Shi Xiaoyu [20] and Yan Zaojun [21] et al. used a two-layer back-propagation (BP) neural network prediction algorithm to predict the open-loop control voltage of the DM, and the correction effect was about 30 times better than that of the RLS algorithm for the same wind speed. In 2020, Chen Ying et al. used Long Short-term Memory (LSTM) network for voltage prediction, and its prediction residual was about 8 times lower than that of BP network [22].
In terms of network input and output types, direct prediction of the wavefront can be more advantageous, because the prediction model is largely unaffected by changes in SHWS and DM, thus, there is no need to design specific network models for different AO systems. Therefore, the way of directly predicting This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ the wavefront has higher compatibility and applicability in most of AO systems. From the perspective of network model, for dealing with the time series prediction problem of wavefront prediction, LSTM network has more advantages under the same environment configuration [23]. This is because LSTM network has a strong memory ability and can accurately learn the temporal features between successive distorted wavefront frames to meet the prediction requirements. However, due to the temporal features of continuous wavefront, the direct feature extraction from the unrefined continuous wavefront will cause redundant errors, which damages the prediction accuracy. Therefore, a deep learning-based RLFNet is proposed for the first time in this paper, which eliminates the redundant information between frames by means of residual learning, then extracts the refined features for fusion, and fuses the features at all levels again to obtain a high accuracy prediction result. The prediction performance of our proposed RLFNet is tested from simulation and experiment results under different types of atmospheric turbulence conditions.

A. Atmospheric Turbulence Distorted Wavefront Simulation
The atmospheric frozen flow hypothesis is a widely adopted modeling approach for atmospheric turbulence that has been verified by a large number of experiments [24], [25], [26], [27]. It has been shown that the time scale on which the frozen flow hypothesis holds is 10∼20 ms, reaching 50∼100 ms in a few cases [28]. In the AO system with a sampling frequency of more than 1 kHz, the time delay of the system is basically less than 3 ms, so the frozen flow hypothesis in such a system is reasonable.
Atmospheric turbulence occurs on a cascade of spatial scales with a power law dependence well approximated by Kolmogorov's Law [29], that is, the refractive index structure function in the inertial region satisfies the "two-thirds law", which is expressed as: Where: r is the scalar distance, C 2 n is the refractive index structure constant changing with altitude h. As a statistical model, the expression of the classical model HV-57 is as follows: Among them, generally take: v = 21m/s, A = 1.7 × 10 −14 . The formula shows that atmospheric turbulence is the strongest at near the ground. As the altitude increases, C 2 n experiences a process that will first become weaker, then stronger, and finally gradually weaken.
In laser atmospheric transmission and AO correction technology, D/r 0 is widely used to describe the influence of turbulence effects. where D is the diameter of the telescope, and the r 0 is the Fried parameter. The larger the value of r 0 , the better the atmospheric conditions. In general, the relationship between r 0 and C 2 n is [30]: Where k is the wavenumber, L is the transmission distance, and ϕ is the zenith angle.
If the horizontal transmission, i.e., ϕ = 0, the above equation can be simplified as: The variation of wind speed with altitude in the atmosphere is often expressed by the Buffton wind speed model, as: Among them, generally take In this paper, the simulation data is generated by the time evolution of the Fourier series method based on the above atmospheric turbulence theory. We simulate the distorted wavefront according to the experimental scene of the 1 km near-ground laser atmospheric near-horizontal transmission system. Since the recovery accuracy of SHWS is limited, the phase screen has more detailed phase structure information than the wavefront recovered by SHWS, therefore, we use the phase screen as the training data of our proposed RLFNet, which can improve the generalization of the network model.

B. Network Model Setup
The principle of our proposed RLFNet applied to the AO system is shown in Fig. 1. The SHWS is used to detect the slope information of the wavefront, and then after wavefront reconstruction, the control signal is output by the control system with prediction model to drive the DM to produce deformation, so as to compensate the actual distorted wavefront. In this paper, we focus only on the implementation of the wavefront prediction neural network model, which belongs to the open-loop prediction process, so the present study procedure is shown by the solid black arrows.
There are five key points in building a supervised network model: input, output, loss function, label, and model structure [31]. There is a trade-off between the network prediction time and the number of wavefront predicted by the output. In order to accommodate to the time delay of 2∼3 sampling periods in most AO systems, in our RLFNet, we use consecutive wavefront of the first n frames as few as possible to predict the wavefront of the n+2 frame. In order to select the most suitable network input, we take n as 3, 4, 5, 6 and 7 respectively. We found that when the input is 6 consecutive frames and later, the accuracy of wavefront prediction remains basically unchanged. Considering that more wavefront inputs tend to bring the problem of excessive computation and the risk of overfitting, we finally selected 6 consecutive frames as the final input of our proposed network. That is, the continuous wavefront from time t to t+5 is used as the input of the network, and the distorted wavefront at moment t+7 is used as the output. Such an input-output setting can effectively avoid both the problem of too much data being processed by the network in a single pass and the wavefront latency for subsequent DM compensation. The loss function of our network model uses the L1_loss function, as follows: Where y(i) predict is the predicted value of the distorted wavefront at moment t and y(i) true is true value of the distorted wavefront at moment t. The L1_loss function is used in our model to minimize the sum of all the absolute value errors between the true value and the predicted value. The smaller the value of L1_loss, the closer the predicted value is to the true value, which means that the prediction effect is better and the residual RMS (RMSe) value between the predicted value and the true value is smaller. Therefore, we choose RMSe as the evaluation index of the prediction performance of our proposed RLFNet.
In our proposed RLFNet, as shown in Fig. 2(a), VGG19 [32] is used as the backbone network, there is a residual learning fusion module (RLFM) in our RLFNet. The redundant information between two adjacent frames is eliminated by means of residual learning, and then the refined features are extracted for fusion. The process is shown as follows: Where σ represents the activate function, i.e., PReLU [33], Concat(. . . , . . .) represents the concatenation operation [34], (ω 1 , b 1 ) denotes the weight and bias of a 1×1 convolutional layer, and " * " means convolutional operation. We use the memory capability of the Convolutional Long Short-term Memory (ConvLSTM) network [35] to predict the output wavefront information at moment t+7 based on the continuous wavefront features from t to t+5, as shown in Fig. 2(b). Finally, the features at all levels are fused again to obtain the final prediction results. The process is shown as follows: Where (ω 2 , b 2 ), (ω 3 , b 3 ), (ω 4 , b 4 ), and (ω 5 , b 5 ) denote the weights and bias of a 1×1 convolutional layer. The final predicted value isP 4 .

A. Data Generating
We use different values of D/r 0 to simulate the distorted wavefront with different atmospheric turbulence conditions, D is set to 1m and the sampling frequency is set to 1 kHz. According to the experimental environment, in the simulation, we set the wavelength to 1064nm and considered five turbulent layers, each of which is evenly distributed within 1km of the transmission path and located in the middle of each segment. At the transmit/receive altitude of about 10 m near the ground, the wind speed is set to about 5.65 m/s, and the wind direction is set randomly 0∼120 degrees to increase the diversity. According to (2), at this time, C 2 n is a certain value, that is C 2 n = 1565e − 17. However, in practice, it will be mainly affected by temperature gradients under different weather conditions [36]. The experimental measurement analysis shows that the variation of C 2 n in the area where our experimental system is located is generally about an order of magnitude [36]. Therefore, according to (4), five increased C 2 n values are selected in this range, that is, 0.1C 2 n , 0.2C 2 n , 0.5C 2 n , 0.7C 2 n and 1.0C 2 n to simulate the atmospheric turbulence intensity under different r 0 conditions. The values of r 0 are 0.152, 0.100, 0.057, 0.047, and 0.038, that is, D/r 0 are 6.58, 10.00, 17.54, 21.28, and 26.32, respectively.
We input the simulated 5 groups of distorted wavefronts under different D/r 0 into our proposed RLFNet at the same time to improve the robustness of the network. For each dataset, we used the continuous wavefront from time t to t+5 as the input to the network, and the distorted wavefront at moment t+7 as the label. 10500 groups of data were generated under each D/r 0 condition, among which the first 10000 groups were used as training frames and the last 500 groups were reserved to test the performance of the model. That is, a total of 50000 groups of data were used as our training dataset. This splitting of the training and test dataset prevents the network model from learning the input information of unknown data in advance. Fig. 3 more clearly represents the input and output settings of the training dataset of our proposed RLFNet.

B. Analysis of Simulation Results
The prediction performance of our proposed RLFNet at different D/r 0 has been tested after training the network model. We obtained the wavefront at the corresponding moment t+7 from the continuous wavefront data at time t to t+5 in the simulated  The RMSe of all frames within the test dataset to characterize the prediction performance of our proposed RLFNet, as shown in Fig. 4. In addition to that, we also employed the structural similarity function (SSIM) to verify the structural similarity of the prediction results with the real wavefront. Table I shows the RMS value of the original distorted wavefront (Truth RMS), the RMS value of the predicted distorted wavefront (Prediction RMS), the RMSe, the ratio of the RMSe to Truth RMS (RMSe/Truth RMS), and the SSIM for each test dataset. This result is based on the average of all 500 frames from each test dataset.
The above results show that the larger the D/r 0 , the stronger the atmospheric turbulence intensity, and the larger the RMSe. But the RMSe/Truth RMS is basically stable at around 2.5%. This indicates that our proposed RLFNet has a well prediction performance of the distorted wavefront at different turbulence intensities.

C. Ablation Studies
We conducted several ablation studies to demonstrate the effectiveness of each component of our proposed RLFNet. The two methods are built by removing the main part of our RLFNet, respectively. The two methods used exactly the same training dataset and test dataset as our RLFNet, the average RMSe value and SSIM under different methods are intuitively shown in Fig. 5.
More specifically, by comparing the results of the first method without residual learning (expressed as w/o residual-learning) with our RLFNet, we can clearly observe that the results of the w/o residual-learning method are less accurate than those of our RLFNet. This fully demonstrates that residual learning can effectively improve the prediction performance. Secondly, in order to verify the validity of ConvLSTM, the ConvLSTM part was removed from RLFNet and the predicted wavefront was obtained by general convolution method to ensure the consistency of input/output dimensions. The method of removing ConvLSTM (represented as w/o ConvLSTM) was compared with our proposed method, and results showed that w/o Con-vLSTM performed worse. This shows that ConvLSTM can use its strong memory ability to well fuse the historical distorted wavefront information with temporal features and obtain better prediction results. The RMSe/RMS_Ture results of these methods are compared in Table II, which further demonstrates that each component of our proposed method contributes to the final prediction results.

A. Analysis of Experimental Results
Our proposed RLFNet performance was tested on data collected by the 1km laser atmospheric transmission system, where the laser source is a cooperative target and the collection method is one-shot open-loop. The schematic diagram of the experimental operating system for laser atmospheric transmission and the experimental setup is shown in Fig. 6. The laser emitted from the cooperative beacons first enters the telescope, which is reoriented by the tilt mirror (TM) and subsequently corrected for aberration by the DM. The beam is divided into two beams by the spectroscope, one of which is directly output to the Charge Coupled Device (CCD), and the other into the SHWS, which measures the aberration. In this paper, we are committed to the prediction of the distorted wavefront, and do not conduct closed-loop correction experiments, so the experimental data are derived from a one-shot open-loop collection of the system, as shown in the red dashed box in Fig. 6. We were provided with two groups of SHWS data with D/r 0 of 7.6 and 13.6, both with a frame rate of 1 kHz and both with a collection time of 10s. The relevant parameters of the cooperative target and wavefront sensor in the experiment, as well as the data estimated for some atmospheric conditions, are shown in Table III. We used the mode method [37] for wavefront recovery of SHWS data. In the recovery process, it was found that when the zernike mode exceeded 104, the sum of the absolute values of the zernike coefficients of each frame remains roughly the same. Therefore, we only applied 104order Zernike mode for wavefront recovery of experimental data according to experimental parameters. Since the beam jitter has been suppressed by the TM based on CCD information during open-loop data acquisition, the aberration of tilt and tip are not considered in the wavefront recovery.
The presentation of the results for the two groups of experimental data is based on a fine-tuned model. Fine-tuning is a process in which a limited portion of a pre-trained network is retrained on a particular dataset to improve its specificity to this dataset [38]. This is because the nature of simulated data is quite dissimilar from our experimental data, in that they are noiseless   TABLE III  THIS IS A SAMPLE OF A TABLE TITLE models of frozen, discrete layers. The training of fine-tuning is very fast due to the fact that the pre-trained network model has learned the laws of feature extraction and successfully identified the frozen flow part of the experimental data. We set aside  1200 frames of data as a test dataset and did not participate in the training process, so the test dataset was not trained in advance. Fig. 7 shows the comparison of RMS of all frames in the two experimental test dataset and the probability distribution of RMSe. Table IV shows the Truth RMS, the RMSe, the RMSe/Truth RMS and the average SSIM for the two groups of experimental data. Again, this result is based on the average of all 1200 frames within the test dataset.

B. Comparative Experiment
In order to verify the prediction performance of our proposed RLFNet method, we conducted two groups of comparative experiments. One group is a non-predictive method and the other is a linear predictive method. The non-predictive method is also done in a one-shot open-loop condition, where only the recently collected wavefront is used to compensate for future frames, regardless of delay. In other words, two-frame latency system in which the wavefront collected at the moment t-2 is used to compensate the frame at the most recent moment t. We also conducted tests under the assumption of three-frame latency system. Similarly, the three-frame latency system uses the wavefront collected at the history moment t-3 directly to compensate for the wavefront at the most recent moment t. The linear predictive method to be compared in this paper is based on the motion-estimation algorithm of module matching (Motion-Estimation-MM). The motion direction of atmospheric turbulence is estimated according to the wavefront recovery image of the reference frame and the current frame, and then the current frame is mobile-processed to predict the image of the future frame [16]. This Motion-Estimation-MM adopts the search strategy and matching criteria of Adaptive Cross Pattern Search (ARPS) algorithm [39].
We use the same test dataset and prediction strategy to test the two methods and compare them with our proposed RLFNet. Fig. 8 shows the RMSe of all the same frames in the test dataset of the two groups of comparison experiments and our proposed RLFNet.
As can be seen from the test results of the comparative experiments: a) Our proposed RLFNet has a well prediction performance of the distorted wavefront for different atmospheric turbulence conditions. It proves that our network has a good generalization capability, however, there is a slight degradation in performance compared with the simulation data, which is due to the uncertainty and randomness error of the experimental data, as mentioned above. b) The prediction performance of our proposed RLFNet and Motion-Estimation-MM is better than the non-predictive method, and our proposed RLFNet performs best because the matching degree between the wavefront aberrations detected by Motion-Estimation-MM and the frozen flow hypothesis is uncertain, so it is difficult to adapt to the changeable turbulence conditions. As shown in the Fig. 9, we compared the average RMSe of all frames in the test dataset for the three methods. In the two-frame latency system, when the D/r 0 is 7.6, the average RMSe of our proposed RLFNet is 0.0237λ, and the average RMSe of Motion-Estimation-MM is 0.0429λ. Compared with the nonpredictive method, the average RMSe of Motion-Estimation-MM decreases about 15.6%, however, our proposed RLFNet decreases about 53.3%. When the D/r 0 is 13.6, the average RMSe of Motion-Estimation-MM decreases about 15.7%, however, our proposed RLFNet decreases about 54.9%. When our network is applied to a three-frame latency system, although the average RMSe decreases, it shows a better performance improvement. When the D/r 0 is 7.6, the average RMSe of our proposed RLFNet decreases about 55.7%, but the Motion-Estimation-MM decreases only about 13.0%. When the D/r 0 is 13.6, the average RMSe of our proposed RLFNet decreases about 54.9%, but the Motion-Estimation-MM decreases only about 15.2%.
It can be seen that our proposed RLFNet performs better when the atmospheric turbulence is stronger, which is due to the fact that when the atmospheric turbulence intensity is stronger, the difference in structural similarity between two adjacent frames is larger, and this is why our proposed RLFNet shows a better performance. When the time delay of the system increases, the difference between the compensated wavefront and the actual wavefront also becomes larger, so our network performs better in the three-frame latency system instead. Therefore, we believe that there will be better performance improvements if the network can predict up to frame t+8, which will be implemented in the following work.

V. CONCLUSION
In this paper, we propose a novel wavefront prediction method based on residual learning fusion network with high accuracy, which aims to solve the latency problem of the traditional AO system. The method eliminates the redundant information between adjacent distorted wavefront frames due to temporal features by residual learning, and then fuses the refined features at all levels to obtain the predicted wavefront. The simulation results show its effectiveness and generalization ability, and its performance is testified from data collected on the 1km laser transmission atmospheric transmission experimental setup. The results show that the proposed method leads to an average RMSe reduction up to 53% for both different atmospheric turbulence conditions compared to the non-predicted method. Proposed method has potential to solve latency problem in a close-loop AO system, which can improve correction bandwidth significantly.