Sequential Subtraction-Based Compressive Single-Pixel Imaging in Complicate Ambient Light

As a special imaging technique, one of the most important advantages of single-pixel imaging (SPI) than conventional imaging method is that it can recover the object image even through turbid medium. In these situations, the noises bring by the turbid medium usually obey special rules in statistics. While in some other applications, SPI is performed in the environment with complicate ambient light, in which no prior information of the environment noise is known. Aiming at this situation, in this work, the frame-by-frame subtraction-based compressive sensing SPI (FFS-CSPI) method with random 0/1 pattern is used to decrease the effect from the unknown ambient light. The noise robustness of the FFS-CSPI method is analyzed and compared with Hadamard CSPI. In simulation and experiment, two kinds of noises, from external light source and from background video, are considered. The results prove that FFS-CSPI with random 0/1 patterns can achieve higher image quality than conventional mean subtraction method and Hadamard CSPI with the same measurement number. Considering the high refresh rate of digital micromirror device when it loads the 0/1 binary patterns, the imaging speed is acceptable. This work will promote the practical applications of SPI in complicated environment.


I. INTRODUCTION
A T present, the single-pixel camera, which derives from compressive sensing [1], and the ghost imaging (GI), which derives from quantum correlated imaging [2], [3], are both classified as the single-pixel imaging (SPI) techniques. As a particular imaging technique, the SPI image can be obtained using a bucket detector (BD) or a single-pixel detector (SPD) and an algorithm for reconstruction. In general, SPI has a lower imaging quality than "point-to-point" imaging systems when the sampling rate is low. Increase in sampling rate can improve the quality of the SPI image. However, oversamples would reduce its temporal resolution and limit its practical application. Increasing the imaging quality and temporal resolution of SPI is one of the most important research tasks [4], [5], [6], [7]. According to the SPI mechanism, the illumination pattern and the reconstruction In the aspect of illumination pattern, the random speckle pattern, the random binary pattern, the sinusoidal pattern, the Hadamard pattern, etc., are used frequently [8], [9], [10]. As we know, the random speckle pattern and the random binary pattern are not the orthogonal pattern. Their corresponding SPI results are attached with background noise, and their imaging quality are lower than that of the sinusoidal pattern and the Hadamard pattern [7]. Besides, the hardware of the light field modulator used in SPI also can affect the image quality and speed of SPI. The commonly used modulators of light field are the spatial light modulator (SLM), the digital mirror device (DMD), the LED array, and so on [11], [12], [13]. Therein, DMD is one of the most popular tools because it can realize the 0-1 binary modulation at a high refresh frequency nearly 20 kHz. The DMD can maintain a high refresh rate when it performs the 0/1 binary modulation. While the refresh rate would decrease when it performs the sinusoidal pattern, the Hadamard pattern, or the random speckle pattern modulations, because it is necessary to perform two or multiple refreshes to achieve one modulation.
In the aspect of reconstruction algorithm, the differential GI [14] and the high-order GI [15] both can improve the imaging quality effectively. The combination of compressive sensing (CS) [1] and SPI (CSPI) can further improve the imaging quality with a lower sampling rate. Subtraction is also an effective operation to decrease various noise. The mean subtraction technique was commonly used in GI to introduce the intensity fluctuations and increase the contrast [16], [17]. The frame-by-frame subtraction technique were also proposed. Yu et al. proposed the complementary single pixel imaging by using double arm detection with two detectors [18]. Then the single-arm complementary compressive imaging scheme was also proposed. It transforms the difference between the adjacent pattern pair into the new measurement matrix, and transforms the difference between the adjacent detected intensity pair into the new detected result, which is called pairwise subtraction algorithm [19]. In 2019, Li et.al. used the sequential-deviations method to replace the subtract in pairs method [20]. Li et.al proposed that by using random measurement matrix, the sequential-deviations method can increase the imaging quality for both the secondorder correlation function and the CS method. Besides, in the Hadamard-based SPI method, the subtraction operation is also used. The Hadamard matrix comprise ±1 values. Two complementary samplings are sequential performed and subtracted for one Hadamard measurement. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ From above introduction, we can see that SPI system has a high flexibility. Therefore, SPI has been used in a wide range of applications, including complicated environment imaging [21], [22], [23], [24], [25], [26], [27], terahertz imaging [28], and 3D imaging [29], [30], [31], [32]. One of the most important advantages of SPI than conventional imaging method is that SPI can imaging an object through the turbid medium, such as atmosphere turbulence and biological tissue [21], [22], [23]. In these situations, the additional noises bring by the turbid medium usually obey the special rule in statistics. The effect of the turbid medium on SPI result can be analyzed by consider the statistic property of the phase or intensity variation caused by the turbid medium. While in some other application situations, SPI should be performed in the environment with the complicate ambient light. In these situation, no prior information of the environment noise is known.
Aiming at this situation and take into consideration the high refresh rate when DMD performs 0/1 binary pattern, in this work, the frame-by-frame subtraction-based CSPI (FFS-CSPI) method with random 0/1 pattern is used to decrease the effect from the unknown ambient background light. The noise robustness of the FFS-CSPI method is analyzed and compared with the Hadamard-based SPI. In the simulation and experiment, two kinds of noises, from another light source and background video, are considered. The results prove that the frame-by-frame subtraction technique can achieve the higher image quality than the commonly used mean subtraction technique. At the same time, the subtraction in sequence method with random 0-1 pattern can achieve the image quality as high as the Hadamard-based CSPI but only need the half measurement number of it. Considering the high refresh rate of digital micromirror device when it loads the 0/1 binary patterns, the imaging speed is acceptable. This work will promote the practical applications of SPI in many real-time requirements and complicated environment with noise.
II. FRAME-BY FRAME SUBTRACTION-BASED CSPI Suppose the N random 0/1 binary illumination patterns are P n (x,y), n = 1, 2, …, N. The object image with M pixels is represented as O(x,y). The nth detected intensity of BD is and its matrix form is in which Y is the column matrix form of B n , n = 1, 2, …, N with the size of N × 1. X is the column matrix form of O(x, y) with the size of M × 1. A is the measurement matrix with the size of N×M. The n th row of A is the row matrix form of the n th illumination pattern P n (x, y). The schematic diagram of (2) is shown in Fig. 1(a). The illumination patterns used in this scheme are 0/1 binary patterns. Therefore, the measurement matrix A is also a binary matrix. For conventional mean subtraction CSPI (MS-CSPI), Y and A is subtracted by the mean value of all the frames and become

Y' and A':
The imaging result is obtained by solving (4) with a range of alternative reconstruction techniques based on greedy, stochastic, variational algorithms, and so on [1], [33].
While in FFS-CSPI, the illumination patterns and the detected intensities of BD are subtracted frame-by-frame, which equals to the differential operation in row, and (2) is turned into diff() represents the differential operation, which is shown as Fig. 1(b). The size of Y FFS is (N-1)×1, and the size of A FFS is (N-1)×M. For the sake of detail, the n th row of (6) is The elements in A are 0 and 1. If the proportions of 0 and 1 in the illumination patterns are both 50%, the mean value of the elements in A is about 0.5. If the proportion of 1 is decreased, the mean value of A would approach to 0. While the elements in A MS are 0.5 and −0.5, and the elements in A FFS are −1, 0, and 1. The mean values of the elements in A MS and A FFS are both approach to 0. According to [18], [19], [20], 0/1 matrix cannot obtain a good result unless the number of 1 is much smaller than that of 0. While the very sparse sensing matrix would decrease the luminous flux, which would future decrease the signal-tonoise ratio. While the subtraction method can increase the image quality. In addition, according to the theory of SPI, the image is recovered because of the fluctuation of the illumination patterns. The highs fluctuation of the illumination patterns can produce the higher image quality. Therefore, the recovered image of (5) would have a higher image quality than that of (2) under the same measurement number and illumination patterns.
If a background noise or the external light source is illuminated on the object together with the designed illumination patterns, and the ambient background light is also detected by

BD, then (2) becomes
in which S represents the noise matrix arise from the effect of ambient light on the illumination patterns and the detected intensities, and the size of it is N×M. k 0 is the parameter to control the extent of noise. Then the nth row of the FFS result in (6) is Equation (9) shows that, if the variation trend of the background is gentle, the second item in (9), k 0 (s n+1 -s n ), is smaller than k 0 s n and k 0 s n+1 . Therefore, the effect of noise on SPI would be decreased effectively.
In the next part, the noise robustness of the FFS-CSPI with random 0/1 patterns are tested and compare it with the Hadamard pattern-CSPI.

A. Without Noise
We first consider the noiseless situation. The simulated object in Fig. 2(a1) contains the horizontal and vertical structure. The sizes of the objects are 128 pixel×128 pixel. The proportions of 0 and 1 in the used random 0/1 patterns are 50% and 50%, respectively. The CS algorithm used in this paper is the TVAL3 (Total Variation Augmented Lagrangian Alternating-direction Algorithm) algorithm [34], [35]. The signal to noise ratio (SNR), peak signal to noise ratio (PSNR), correlation coefficient (CC), and structural similarity index measure (SSIM) values are used to evaluate the image quality.   Fig. 2(c1) is higher than Fig. 2(d1) visually. This comparison is more obvious in Fig. 2(c2) and 2(d2), in which the object is an irregular structure. Fig. 2 prove that the FFS operation can achieve the higher image quality than MS operation when the illumination patterns are random 0/1 patterns. In addition, the FFS-CSPI can obtain a comparable image quality with Hadamard pattern but only half of the illumination number is needed.

B. With Environment Noise
Then we test the noise robustness of FFS-CSPI. According to the theory of SPI, the detected intensity of BD can be treated as the transformation coefficient of the object image under the corresponding transform model. For the random 0/1 models, the corresponding transformation coefficients are more uniform than that of Hadamard models, which is shown in Fig. 3. Therefore, on the one hand, the analysis in Section II shows that the FFS operation can effectively decrease the effect of external light. On the other hand, when the illumination number is small, the noise robustness of Hadamard-based SPI against noise must higher than that of random 0/1 pattern-based SPI, especially for the high frequency noise.
The first simulated external environment noise is a light source which can emit the flicker light with the periodic changing intensity and the changing rule conforms to the cosine curve. The value of the n th row of S in (8) is cos(2πn/t 0 ), where t 0 is used to control the flicker periodic of the light source. In theory, a smaller periodic would lead to the higher flicker frequency, and the imaging quality would be decreased. Moreover, the increase of k 0 would also decrease the imaging quality. Fig. 4 show the recovered images for Hadamard pattern and random 0/1 pattern with different k 0 and t 0 . Firstly, we can notice that the recovered images of the random 0/1 pattern MS-CSPI are blurred by the noise and no information can be obtained from Fig. 4(c1)-(c4). The FFS operation can enhance the imaging quality and the structure of the object is clearer than that of Fig. 4(c1)-(c4), which can be seen from Fig. 4(d1)-(d4). Secondly, we can find that the imaging qualities for both Hadamard pattern and random binary pattern are decreased with the increase of k 0 and the flicker frequency 1/t 0 of the external light source. Thirdly, Fig. 4 show that the downtrends of the Hadamard pattern and the random 0/1 pattern are different. When the flicker frequency of the light source is small, the FFS-CSPI can achieve better performance than Hadamard SPI when the illumination number is the same. While with the increase of flicker frequency, the image quality of FFS-CSPI is decreased more quickly than that of Hadamard SPI.
For more details, Fig. 5(a) show the curves of CC values versus t 0 when k 0 = 1. Fig. 5(b) show the curves of CC values versus k 0 when t 0 = 100. Fig. 5(a) show that the FFS-CSPI method is more sensitive to the flicker period t 0 than the Hadamard-based CSPI and the imaging quality is decreasing rapidly when the flicker period is decreased (which equals to the increase of the flicker frequency). To the contrary, Fig. 5(b) show that when  the flicker period is large, the downtrend of the FFS-CSPI method is gentle, which means it has a higher robustness to the intensity k 0 of external light source than Hadamard-based CSPI method when the flicker period is large. Even so, we can foresee that if the flicker period t 0 is decreased, the flicker frequency of the external light source is increased, the decreasing trend of the CC value versus k 0 would become faster than that of Fig. 5(b).
The second simulated external background noise is a video with different frame rate that equals to the 1/21, 1/9, and 1 refresh rate of illumination patterns. The video is "rhinos.avi" in MAT-LAB R2021b. Here k 0 is the parameter to control the intensity of the video. Fig. 6 show the CSPI results with Hadamard patterns and random binary patterns when k 0 = 2. The corresponding SNR and CC values are calculated.
When the frame rate of the video is small, FFS-CSPI can maintain the high image quality, which is shown in Fig. 6(c1)  Fig. 7. Experiment setup. TIR prism is total internal reflection prism. f 1 = 5 cm, f 2 = 20 cm, f 3 = f 4 = 10 cm. and 6(c2). While when the frame rate of the video is high, Fig. 6 show that the Hadamard-CSPI is more stable than the FFS-CSPI. This is still consistence with Figs. 4 and 5.
The simulation results above prove that FFS-CSPI has a high noise robustness against the complicate environment light especially. In consideration of the easy availability, less restrict, and high refresh speed of the random 0/1 pattern than Hadamard pattern, the combination of the simple FFS-CSPI and random 0/1 patterns can be applied in further more SPI application scenes.

IV. EXPERIMENT RESULTS
To verify the simulation results, a series of experiments for random 0/1 pattern and Hadamard pattern on the SPI system are implemented. The SPI experimental setup is illustrated in Fig. 7. A 632.8 nm laser beam is expanded by L 1 (f 1 = 5 cm) and L 2 (f 2 = 20 cm) and thereafter modulated by a DMD. The size of the DMD is 1920 pixel × 1080 pixel, with each pixel of size 7.56 μm × 7.56 μm. The maximum binary pattern refreshing rate of the DMD is 9 kHz. The reflected beam from the DMD is projected onto the object plane using a lens (L 3 , f 3 = 10 cm). The object used in this experiment is the Negative 1951 USAF Target (1-228 LP/mm). The transmitted light from the object is converged by L 4 (f 4 = 5 cm) and thereafter detected by the BD (PDA 100A2, Thorlabs). The detected signal is transmitted to the computer through the data acquisition card (DAQ, NI USB-6341).
In the experiment, the random 0/1 patterns (with the proportions of 0 and 1 are both 50%) and Hadamard model patterns with 128 pixel × 128 pixel are used. In DMD, 8 pixel × 8 pixel are treated as a new pixel. Therefore, 1024 pixel × 1024 pixel is used in this experiment. One Hadamard model measurement is acquired by projecting two complementary "0-1" distributions on object.

A. Experiment Without Noise
When the experiment is performed in the dark room, the results with Hadamard patterns CSPI when N = 4000 and 8000 are shown in Fig. 8 We can see that the resolution of the FFS-CSPI with N = 4000 random binary patterns is higher than the Hadamard-based CS-SPI with N = 4000 patterns, and is comparable with the Hadamard-based CS-SPI with N = 8000 patterns. These results are consistent with the simulation results in Fig. 2.

B. Experiment With Noise
Then the robustness against background noise is analyzed. When the SPI process is implemented under a fluorescent lamp, the light from the lamp would decrease the image quality of SPI. The sampling rate of the BD is 500kHz. The flicker frequency of the lamp is about 100Hz. The exposure time and dark time of the DMD is 300 μs and 200 μs, respectively, which means 20 times measurement is performed in one lamp flicker period. The recovered images of Hadamard CSPI are shown in Fig. 9(a)-(b).  We can see that the effect of the noise on the Hadamard-based CS-SPI is quite small. The recovered images of random 0/1 pattern MS-CSPI is shown in Fig. 9(c), which is blurry and no information can be obtained from it. Figs. 8(c) and 9(c) also prove that the conventional CSPI based on random 0/1 pattern has a low noise robustness. Fig. 9(d) show the recovered image of the FFS-CSPI by using the same illumination patterns with Fig. 9(c). Fig. 9 show that FFS operation can decrease the effect of noise and improve the image quality obviously, and the noise robustness is slightly lower than Hadamard-based CS-SPI because of the high flicker frequency of the fluorescent lamp.
We also test the robustness against background video. A mobile phone is placed beside the object and play a black-and-white film with the frame rate of 24 fps, the schematic diagram is shown in Fig. 10. The light from the mobile phone is oblique illuminated on the object and then collected by L 4 together with the signal light. The corresponding recovered images are shown in Fig. 11. Different from the high-frequency flicker of the fluorescent lamp, the frame rate of the movie is relatively low compared with the sampling rate. Therefore, the effect of the background video on the SPI based on differential operation is further decreased compared with Fig. 9.
From Figs. 9 and 11 we can see that, when the environment of SPI system is complicate and high-frequency or low-frequency background noise is exist, the FFS operation can observably improve the image quality of random 0/1 pattern-based CSPI. In addition, when the frequency of the noise is relatively low, the proposed method can achieve a comparable image quality with only half of the measurement number compared with Hadamard CSPI. These experiment results are consistent with the simulation results.

V. CONCLUSION
In summary, we consider the situation that the SPI process is performed in a complicate environment with unknown nonrandom ambient light source. Combined with the simple FFS operation, the image quality of the CSPI by using random 0/1 pattern can be improved effectively. The simulation and experiment results prove that the image quality of the FFS-CSPI can obtain the image quality as high as the Hadamard CSPI, but only half measurement number of the Hadamard-based CS-SPI is needed. The noise robustness is analyzed and prove that the FFS-CSPI is sensitivity to the frequency of the disturb. When the frequency of the external disturb is low, the FFS-CSPI method still can maintain its advantage and obtain a better result than the Hadamard CSPI with the same measurement number. When the frequency of the external noise is increased, the imaging quality of the FFS-CSPI by using random 0/1 pattern would decrease faster than that of Hadamard CSPI. Even so, the image quality of CSPI with the FFS method is still much higher than that without FFS operation.
In addition, as we know, the random 0/1 patterns are not the orthogonal pattern. Their corresponding SPI results are attached with background noise, and their imaging quality are lower than that of the sinusoidal pattern and the Hadamard pattern. However, there are some advantages of the random 0/1 binary pattern. For example, it is easy available and less limitation than Hadamard pattern. The rate of DMD to refresh 0/1 pattern is also higher than Hadamard pattern and other gray pattern. This work proves that the simple FFS operation combines with the random 0/1 patterns can not only improve the image quality effectively, but also has a high noise robustness against complicate environment light. We believe that this work will promote the practical applications of SPI in many real-time requirements and complicated environment. Even the FFS-CSPI method has the advantages above, there still exists some questions to be solved. We can see that the image quality of the FFS-CSPI method is decreasing fast with the frequency of noise. If the frequency of the ambient light is approach to or even larger than the frequency of DMD, the FFS-CSPI method would not achieve an acceptable image quality. Maybe we can remove the effect of ambient light by analysis its specific rule in statistic in our future work. In addition, the structures used in this work are relatively simple. The gray scale object or three-dimensional object would be considered in the future.