Adaptive Correction Method of Uneven Intensity in Fourier Ptychographic Microscopy

Fourier ptychographic microscopy (FPM) is a computational imaging technique that combines a large field of view and high-resolution. It stitches low-resolution images captured under varying illumination corresponding to different angles in the frequency domain to realize quantitative phase imaging with high spatial bandwidth product. Our previous algorithm (AA algorithm) effectively solves the problem of noise interference and the algorithm is difficult to converge. In FPM experiments, the fluctuation of LED intensity is inevitable, which significantly affects the quality of reconstructed images. The usual solution is to perform intensity correction in the reconstruction algorithm. However, in this study, it is found that the AA algorithm is incompatible with the intensity correction method, which will decrease the convergence performance. To solve this problem, we develop a more general reconstruction algorithm based on the AA algorithm in this study. The simulation and experimental results show that the improved algorithm framework is compatible with the intensity correction method, resulting in an increase in intensity correction reliability and effectiveness. Compared with the GS and AS methods with intensity correction, the error in reconstruction results can be reduced by 52.03% and 29.35%, respectively, without increasing the amount of calculation.

the sequentially acquired images are stitched. This is the easiest way to achieve high SBP imaging. However, its application is limited by the high cost and time-consuming acquisition; further, maintaining the focus of samples in a small depth of view is technically challenging [4].
Fourier ptychographic microscopy (FPM) is considered as the best alternative to achieve high SBP imaging and has become a widely used microscopic imaging tool [5], [6], [7], [8]. FPM can provide quantitative phase information of samples without operations such as staining and fluorescent labeling [9]. This is attractive in transparent biological sample imaging because it does not change or affect the activity of living cells. Another advantage of FPM is that it can be built in the hardware part of the microscope with ease, without the need for additional high-cost hardware [10]. The simplest way to build an FPM system is to replace the microscope source with an LED array. In a typical FPM system, LED arrays are used to irradiate samples from different incident angles, and low numerical aperture (NA) objective lenses are used to obtain corresponding low resolution (LR) images. At each illumination angle, the captured LR image corresponds to the information intercepted by the circular aperture in the sample Fourier space.
The core of FPM is to splice all LR images by iterative phase recovery method in Fourier space and then recover information beyond the cutoff frequency of the objective lens. According to previous studies, the intensity distribution of LR images is affected by many types of errors such as LED intensity fluctuations and random noise. When the error exceeds a certain range, it leads to the convergence error of the reconstruction algorithm [11]. Among them, the noise mainly comes from the dark field images captured under high-angle illumination, which usually have low signal-to-noise ratio. Currently, a few improved algorithms have been proposed to minimize the negative impact of noise in FPM reconstruction [12], [13]. To further optimize the impact of LED intensity instability, Bian et al. corrected the intensity fluctuation in the optimization process of a reconstruction algorithm and achieved a good intensity correction effect [14]. In addition, this correction method can be combined with other reconstruction algorithms, such as Gerchberg-Saxton (GS) [5], [15] and Adaptive step-size (AS) [16] algorithms.
In our previous work, the conventional FPM reconstruction algorithm was improved, and it is referred to as AA algorithm in this paper [17]. AA algorithm has been successful in improving the reconstruction quality and convergence, and the effectiveness of AA algorithm in reconstructing high-resolution (HR) images This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ has been verified for most cases. However, the reconstruction error when the LED intensity fluctuates was not discussed. In the experiment, the LED intensity fluctuation directly affects the captured LR image quality, thus affecting the final reconstruction result. In this study, the influence of LED intensity fluctuation on image reconstruction is discussed, and the incompatibility of the intensity correction method with the AA algorithm is addressed.
The AA algorithm is optimized, and the optimal regulation factor in the process of updating the spectrum function is calculated by adding a suitable threshold judgment. The simulation and experiments prove that the improved method is feasible and effective for LED intensity correction, and that it significantly reduces the negative impact of LED intensity fluctuation. Compared with other algorithm frameworks (GS and AS) combined with intensity correction, the improved method in this study achieves better imaging performance.

A. Principle of AA Algorithm
The typical FPM imaging model is mainly composed of an LED array, sample, low NA objective lens, and camera, which is described in detail in [5]. The FPM imaging process includes image acquisition and reconstruction. In the image acquisition module, the LED array is placed on a horizontal plane at a certain height below the sample, and the LEDs are sequentially lit to provide illumination at different angles. At the imaging end, the camera is used to capture the corresponding LR image as the original image data. When the sample is irradiated from different angles, a series of LR images captured correspond to the shift of the pupil function in the Fourier spectrum of the sample. Suppose that the transmission function of the thin transparent sample is s(x, y) ,it is illuminated by an oblique plane wave with a wave vector from LED m,n (m row, n column), and the spectral distribution at the back focal plane of the microscope objective can be expressed as: where S(u, v) is the spectrum of the sample, P(u,v) is the pupil function of the imaging system, which can be regarded as a low-pass filter, and (u, v) are the coordinates in the frequency domain. It can be seen from (1) that under the irradiation of the oblique plane wave, the high-frequency information originally exceeding the cutoff frequency can be transferred to the passband of the system and then received by the camera. The intensity image acquired at the imaging end can be described as: where F −1 represents a two-dimensional inverse Fourier transform operation. The LR images obtained from different illumination angles constitute the amplitude constraint condition of FPM in the spatial domain. FPM stitches these images in the frequency domain through iteration, and then widens the spectrum range of the image. The implementation process of the AA algorithm is described below.
The first step is initialization. Before running the iterative algorithm, the initial spectrum guess of the sample HR complex amplitude distribution and pupil function of the imaging system are constructed, where the pupil function remains unchanged in the iterative update process.
The second step is the interception process, which uses the pupil function to intercept a sub region from the initial HR spectrum and transforms it into the spatial domain to generate a target complex amplitude image I t m,n exp(iϕ t m,n ), where t represents the target image, and m and n represent the index positions of the LED. This process is equivalent to low-pass filtering of the imaging system, and the intercepted sub region corresponds to the spectrum information of the LR image captured by the LED m,n illuminated samples.
The third step is the noise removal process. The noise threshold is obtained by calculating the difference of the arithmetic mean intensity between the target image and the actual measurement image. The threshold calculation formula is as follows: where I m,n and I t m,n represent the measurement image and the target image, respectively. Noise reduction is then performed by processing by subtracting a noise threshold from the measurement image as follows: where, I r m,n represents an image after noise reduction processing. The denoised measurement image is used to update the amplitude component of the target image.
The fourth step is the replacement process. This comprises of using the denoised measurement image I r m,n obtained in the third step to replace the amplitude informationI t m,n of the target complex amplitude image. Furthermore, the phase information remains unchanged, and the updated target complex amplitude image I r m,n exp(iϕ t m,n ) is obtained. The fifth step is to Fourier transform the updated target complex amplitude to obtain the updated HR spectrum distribution and calculate the difference between the spectrum distribution before and after the update to update the HR spectrum information in the corresponding sub aperture of the sample. The AA algorithm introduces an adaptive control factor α value to balance the update process of the sample spectral function, which can significantly improve the image reconstruction quality and convergence speed, where the α value is determined using the following formula: The α value introduced in the AA algorithm is determined from the image noise reduction threshold level. AA algorithm provides a new form of spectrum function update, as shown in (6): where S n (u) represents the HR sample spectrum function in the corresponding sub aperture obtained in the nth iteration. W 1 is defined as the ratio of the current value to the maximum value of the pupil function, δ1 is the regularization parameter to ensure numerical stability, Δφ n (u) represents the difference in spectrum distribution before and after the sample update.
The sixth step is to repeat steps 2 to 5 for other different illumination angles until the spectrum information in all sub apertures has been updated, which marks the completion of one iteration.
The seventh step, repeating steps 2 to 6, continues the iterative update process until the termination condition of the iterative algorithm is satisfied.

B. Conflict Between Intensity Correction and AA Algorithm
For the FPM imaging system, the factors that affect whether the image reconstruction resolution can be closer to the theoretical value mainly include the stability of the device and the effectiveness of the reconstruction algorithm. The stability of the device will directly affect the quality of the captured LR image, for example, the nonuniformity of LED intensity and the random noise of the dark field image under high angle illumination. The AA algorithm combines the advanced improved threshold method to denoise LR images [13], however the reconstruction error when the LED intensity fluctuates is not discussed. Currently, physical and algorithm corrections are used to solve intensity fluctuation problems. Among them, algorithm correction has been widely used because of its simple operation and excellent performance. Bian et al. first proposed an adaptive correction scheme, which evaluates the convergence effect by introducing reconstruction quality indicators into the reconstruction algorithm framework and corrects errors such as intensity fluctuations in the iterative process.
Bian et al. modified the intensity fluctuation in the optimization process. In the first iteration, they followed the reconstruction algorithm flow to obtain better sample estimation. Starting from the second iteration, a group of intensity correction factors are introduced to compensate for the intensity error of the LED: where c m,n represents the intensity correction coefficient corresponding to the image I r m,n . At the end of the second iteration, a set of intensity correction factors c m,n are calculated. In the replacement process of the fourth step of the reconstruction algorithm, c m,n is used to update the original measurement image I r m,n = c m,n · I r m,n . In the subsequent iteration process, the intensity correction factor is constantly updated until the deviation of c m,n is less than the preset value. It should be noted that the intensity of each measurement image is corrected according to its respective coefficient c m,n .
The conflict between LED intensity fluctuation and AA algorithm convergence is verified by simulation, and the influence of AA algorithm on HR image reconstruction with and without intensity correction is compared. The parameters of the simulation experiment are as follows: a 15 × 15 LED array is used to provide illumination at a position 86 mm below the sample, the spacing between adjacent LEDs is 4 mm, the LED incident wavelength is 632 nm, the objective lens NA is 0.1, the magnification is 4 times, the pixel size of the camera is 6.5 μm × 6.5 μm. The "cameraman" and "Westconcordorthophoto" images (256 × 256 pixel) were used as initial input HR intensity images and phase images, respectively. The comparison of experimental results is shown in Fig. 1. Fig. 1(a) and (b) show the reconstruction results of the AA algorithm with and without intensity correction under the combined influence of 20% random noise and 60% intensity fluctuation. The comparison results show that the AA algorithm with intensity correction can improve the image reconstruction quality to a certain extent, and the image background becomes cleaner. However, when comparing the effects of different degrees of intensity fluctuations on the reconstruction quality, it is found that the AA algorithm with intensity correction is unstable. As shown in Fig. 1(c1), the relationship between the root mean square error (RMSE) of the reconstructed HR intensity image and intensity fluctuations of different degrees, shows that the quality of the reconstructed intensity image has noticeable fluctuations. The reconstructed phase image quality shows a relatively stable change as a whole, as shown in Fig. 1(c2). Therefore, the simulation results show that there is uncertainty conflict between the intensity correction and AA algorithm.

C. Improved Algorithm Principle
Since the optimal control factor α value of the AA algorithm is calculated at the threshold level of image noise reduction, the interference of intensity fluctuations is negligible. When there are fluctuations in intensity, the α value calculated in (5) is not an optimal regulatory factor. Therefore, in the version of the improved AA algorithm, the comprehensive consideration of the noise reduction threshold and the intensity correction coefficient is added. Fig. 2 shows the implementation flow chart of the algorithm. The left side shows the framework of the conventional AA algorithm, and the right side includes intensity correcting module and adaptive α value selection module.
When the FPM reconstruction algorithm is executed, the first iteration is carried out to follow the algorithm flow and obtain a better sample estimate. It should be noted that the intensity correction is not performed in the first iteration, and the calculated intensity correction coefficient is only used for the next step of judgment. The optimal regulation factor α value is calculated using (5). Before performing the second iteration, the standard deviations T1 and S1 of the noise reduction threshold and the intensity correction coefficient obtained in the first iteration are calculated respectively. The relative sizes of T1 and S1 are then compared. If T1 > S1, use (5) to calculate the best regulatory factor and perform the subsequent algorithm reconstruction. On the contrary, (8) is used to calculate the best regulatory factor and perform the following algorithm reconstruction. Starting from the second iteration, an intensity correction factor is introduced to compensate for the intensity error of the LED.
In addition, this paper also defines the uniformity C of the intensity correction coefficient, which is used to characterize the stability of the intensity correction coefficient. The smaller the value and the faster it tends to be stable, the better the intensity correction effect of the algorithm. In the current iteration, the calculation formula of the uniformity C of the intensity correction coefficient is as follows:

A. Simulation Experiment
In this section, simulation experiments are carried out to further verify the effectiveness of the improved method for intensity correction. The model parameters used in the simulation are consistent with those in Section II.B above. In this simulation experiment, 225 LR images were generated, and FPM reconstruction was performed on these images. In this simulation, 20% random noise and 40% intensity fluctuation are introduced into the original image, and 21 iterations are set for all reconstruction algorithms, which can meet the convergence conditions required by the algorithm. Fig. 3 shows the comparison of the reconstruction results of the five algorithms. The intensity image and the phase image reconstructed by the AA algorithm without intensity correction are shown in Fig. 3(a1) and (a2), respectively. It can be found that AA algorithm cannot alleviate the impact of intensity fluctuation, and serious artifacts appear in the background of the reconstructed intensity image, which significantly affects the clarity of the image. Subsequently, intensity correction is added to the AA algorithm, and the reconstruction results are shown in Fig. 3(b1) and (b2). It can be observed that the artifacts of the image are eliminated to some extent, however the blur degree of the image is increased. This indicates that the regulatory factors used in the AA algorithm are not optimal, which have been discussed in our previous work [17]. As a comparison, the reconstruction results of the GS and AS algorithms (both with intensity correction) are also given. As shown in Fig. 3(c), the background obtained by using the GS algorithm with intensity correction is more blurred. This cannot meet the requirements of high-quality imaging. Fig. 3(d) shows the reconstruction results using the AS algorithm with intensity correction. The results show that this method can reduce the influence of image blur to a certain extent and presents better reconstruction quality than AA and GS algorithms, however there a subtle background interference remains. In addition, as the algorithm adopts an adaptive step size update strategy, the convergence performance of this method will be limited to a certain extent. Finally, it is shown that the improved method can significantly improve the reconstruction quality and obtain better results with minimal background noise. The intensity and phase images recovered by the improved method are shown in Fig. 3(e1) and (e2).
In addition, the uniformity of the intensity correction coefficient was further quantitatively analyzed, and the results are shown in Fig. 4. As shown, for the AA algorithm without intensity correction, corresponding to the curve of case 1 in Fig. 4, the intensity correction coefficient does not change at this time. This is because only the intensity correction coefficient is calculated in the reconstruction algorithm, and this coefficient is not used for intensity correction. The curve of case 2 in Fig. 4 corresponds to the AA algorithm with intensity correction. The results show that the method has a certain intensity correction effect; however, the overall convergence is poor. For the GS algorithm with intensity correction (case 3), although it can reach  a stable value at a fast speed, the overall value is larger, which indicates that the intensity correction effect is poor at this time. Finally, the uniformity of the intensity correction coefficient with the intensity correction as algorithm (case 4) and the improved method is compared. Although the two methods finally get close convergence values, the intensity correction coefficient obtained by the improved method in this paper can converge to a stable value faster, which shows that the method has a faster convergence speed.
The RMSE value and running time of the reconstructed images by different methods are shown in Table I. The results show that in the presence of uneven LED intensity, the improved method can significantly improve the quality of FPM reconstruction without affecting the operating efficiency and computing time of the original reconstruction algorithm. Compared with the GS and AS methods with intensity correction, the improved method can reduce the error of the reconstruction results by 52.03% and 29.35%, respectively.

B. Real Experiment
In this section, the open-source biological sample data provided by Zheng et al. are used for further experimental validation [18]. The acquisition of this dataset is based on the following experimental platform: an image sensor with a pixel size of 2.4 μm × 2.4 μm, a 32 × 32 LED array with a pitch of 4 mm and an incident wavelength of 531 nm, placed 90.88 mm below the sample. During the experiment, 15 × 15 LEDs were sequentially lit in a spiral outward manner, and a total of 225 LR images were collected. Fig. 5(a) is the raw image captured under central LED illumination. The intensity and phase images are reconstructed using the GS algorithm and the AS algorithm with intensity correction, and the improved method in this paper. The results are shown in Fig. 5(b)-(d). To better show the effectiveness of the improved method in improving the reconstruction quality, the parts of two different regions of the reconstructed phase of the three algorithms are enlarged and displayed, as shown in the green dashed box in the figure. Fig. 5(b3) and (b4) show the phase distribution of the two small regions reconstructed by the GS algorithm. It can be found that the reconstruction result is fuzzy and cannot distinguish the detailed information of cells. Fig. 5(c3) and (c4) are the results of reconstruction by the AS algorithm. Compared with the GS algorithm, the reconstruction quality has been greatly improved, however the image background still has noticeable noise interference. The cell structure information reconstructed by the improved method in this paper is clearer, and the noise interference problem in the image background has been significantly improved, as shown in Fig. 5(d3) and (d4).
To verify the effectiveness of the improved method proposed in this study, the GS algorithm with intensity correction, the AS algorithm with intensity correction and the improved method are used to reconstruct the HR image of the USAF resolution board. A common optical microscope (magnification 4×, NA = 0.1) was used to build an FPM imaging system, and an 11 × 11 LED array (4 mm pitch, 531 nm incident wavelength) was used to replace the microscope source to provide illumination with varying angles, the sample is placed 56.2 mm away from the LED array. LR images from different illumination angles were captured using a scientific-grade camera with a pixel size of 2.4 μm × 2.4 μm. In the experiment, 121 LR images were captured using this experimental platform and used to perform FPM reconstruction. The full field LR image of the USAF resolution plate captured by the objective lens is shown in Fig. 6(a). A small selected area in the full field of view (400 × 400 pixels) is used for result comparison as shown in Fig. 6(b1). Fig. 6(c1) and (d1) show HR intensity images reconstructed using the GS and AS algorithms (both with intensity correction), respectively. It can be found that the reconstruction results of these two methods have   Fig. 6, the lines of element 2 in the seventh group of the reconstruction result are significantly distorted and folded, which are enlarged for comparison, as shown in Fig. 6(c2) and (d2). Among them, the reconstruction result of GS algorithm is the worst, and the image background has the most significant artifacts. Fig. 6(e1) shows the result of reconstruction using the improved method in this paper, and the image background is cleaner. Although there are still some artifacts, in contrast, the reconstruction quality has been greatly improved, the lines are clearer, and there is no distortion and wrinkle, as shown in Fig. 6(e2).
Finally, the final error in the reconstruction process is taken as the evaluation index of imaging quality, and the running time of different methods is compared. As shown in Table II, the final error of GS algorithm with intensity correction is at a high level, indicating that the reconstructed image quality is the worst. The AS algorithm with intensity correction improves the reconstruction quality, but the running time is the longest. This is because the algorithm needs to continuously reduce the step size in the optimization process to achieve high-quality reconstruction. In contrast, the improved method in this paper has a great advantage in running time, and the final reconstructed image quality is better than the AS algorithm.

IV. CONCLUSION
In conclusion, we proved the conflicting relationship between LED intensity fluctuation correction and convergence of the AA algorithm through experiments. Furthermore, we made improvements to solve the incompatibility problems between the conventional intensity correction method and the AA algorithm. Compared with the GS and AS algorithms with intensity correction, the modified solution significantly improves the effectiveness and stability of the reconstruction algorithm for random intensity fluctuation correction and reduces the occurrence of errors in the reconstruction process. The simulation and experimental results showed that the improved method discussed in this paper maintains the good convergence performance and high efficiency of the conventional AA algorithm, and effectively improves the quality of the reconstructed image.