The Extreme Learning Machine for the Extraction of Mono-Exponential Decay Time

The mono-exponential decay has been used to describe various physical phenomena such as cavity ring-down signal, fluorescence decay, etc. In this paper, a neural network method of extreme learning machine (ELM) is adopted to efficiently extract decay time. The theoretical extraction precision, accuracy, and computation cost are all preliminarily analyzed and quantitatively compared with the traditional Levenberg-Marquardt (LM) algorithm. The training set and the testing set for the ELM are built based on our experimental system parameters. After dataset training, the ELM model for mono-exponential decay extraction is obtained. In the dataset testing, this model gives almost the same results with the LM algorithm. The relative deviation of precision is only about ±1 nanosecond. This ELM model can also be directly used in experimental cavity ring-down system. Comparing with the LM algorithm, the relative deviations are less than ±2 nanoseconds when the decay time is in the range of 0.98 μs∼2.20 μs. The ELM method for mono-exponential decay extraction has high efficiency and fine robustness. It has the potential for the applications in cavity ring-down spectroscopy, fluorescence decay analysis, and nuclear radioactive technique, etc.


I. INTRODUCTION
M ONO-EXPONENTIAL decay signal is quite common in cavity ring-down spectroscopy [1], [2], nuclear radioactive decay [3], and fluorescence decay analysis [4], etc. For a mono-exponential decay, the decay rate is mainly focused because it takes the information for research. The decay time, which is the time interval from signal initial intensity to the 1/e of its initial intensity, is a typical measure of decay rate. It is of great importance to precisely and accurately extract decay time from the decay signal with background noise.
As introduced in [5], the extraction of decay time can be qualified by three requirements that are the accuracy, precision, and computation cost. Several methods have been introduced for extraction of decay time, such as the weighted least square (WLS) fitting algorithm [6], the nonlinear least square (NLS) fitting algorithm [7], and the linear regression of sum (LRS) method [8], etc. The extraction performances of these methods have been studied by several papers [5], [9], [10], [11], [12], [13]. Theoretical comparisons show that the nonlinear least square fitting method (typically the Levenberg-Marquardt algorithm) has the best extraction precision so far [10], [11], [12]. Its extraction accuracy is also highly fine. Therefore, the Levenberg-Marquardt (LM) algorithm is deemed as the defacto standard [14] of the mono-exponential decay time extraction. The LM algorithm is an iterative optimization method. It needs to set proper initial values to avoid falling into wrong local optimization. Furthermore, it has been found that the data-point truncation has considerable effect on its decay time extraction [11]. The LM algorithm needs many data-points to achieve high precision and fine accuracy. So the LM algorithm is generally considered to be time-consuming [15], [16] because its large computation cost. For efficient extracting of decay time, a neural network method of extreme learning machine (ELM) is introduced for its robust nonlinear fitting ability [17], [18]. The ELM method can obtain the nonlinear fitting model in certain noisy mono-exponential system. After dataset training, the ELM method can extract the decay time with high precision and accuracy without relying on the iteration and preset initial values. It needs much less data-points than the LM algorithm, which can help to reduce the computation cost and increase the frequency of decay signal processing. The ELM method is tested by simulations and experiments, respectively. Its extraction performance is also compared with the LM algorithm. Its extraction results are nearly same to the LM algorithm. The relative deviation is less than ±2 nanoseconds.

II. PRINCIPLE
For a mono-exponential decay signal, its mathematical expression can be given by (1) Here, S is the mono-exponential signal, A is amplitude, τ is the decay time constant, and B is the combination of instrumental offset and background noise. Generally, B is normally distributed and can be described as B ∼ N(μ B ,σ B ). Here, μ B is the mean value of B and σ B is the standard deviation of noise. For a monoexponential decay, the signal-to-noise ratio (SNR) is defined as This signal is usually digitalized which can be written as In (2), n refers to the nth data-point of S, Δt is the time interval of data acquisition, and N is the total number of data-points. If (2) is fitted by the LM algorithm, the theoretical fitting precision and fitting deviation are given by [9], [10] It should be noticed that both (3) and (4) are only valid if data-points are enough more. In actual applications, the limited N will decrease the fitting performance of the LM algorithm. The neural network model for mono-exponential decay time extraction is built as Fig. 1. The input layer is constructed by the mono-exponential signal and the output layer is the decay time.
In Fig. 1, w means the connection weight between the input layer and the hidden layer, b is the threshold of the hidden layer, and β is connection weight between the hidden layer and the output layer. The subscript i and j refer to the neuron index of the input layer and the hidden layer, respectively. Here, the ELM model can be mathematically described as [17] Here, P is the amount of hidden layer neurons, Q is the amount of training sets, and g(x) represents the activation function. S k means the kth mono-exponential decay signal in the training set. Based on the training set, the ELM model firstly generates random weight values of w and b. The output of the hidden layer H is in the form of Secondly, the connection weight β in the ELM model can be obtained according to Here, H † is the generalized inverse matrix of H, andτ is a matrix including Q decay time constants.
Finally, the ELM model is obtained after dataset training. Based on the ELM model, the decay time can be calculated after the decay signal is inputted. The extraction procedure is descript by Here, the subscript means the dimensions of the corresponding matrix.
The extraction precision of the ELM model can be calculated according to the error propagation formula A way to calculate the extraction deviation is the perturbation theory which is based on the high-order Taylor expansion [11] The <•> in (10) represents the ensemble average. What should be mentioned is that here the extraction deviation and precision is just derived according to the ELM model. Actually the training of the ELM model is along with the noisy decay signal, so the ELM model is also noisy. Therefore the actual deviation and precision will degenerate to certain degree. The training of the ELM always has a little randomness, so its deviation and precision is tested by statistical results of the simulations and experiments in this paper.
The computation cost, which is evaluated by the amount of float point calculations, can be analyzed according to (8). It should be mentioned that the computation cost of dataset training cannot be ignored. Therefore, we recommend the mean computation cost of the ELM method as Here, C represents the mean computation cost, C t is the computation cost of dataset training which is the main computation cost of the ELM method, N s is the number of decay signals, and C g is the computation cost of activation function. The computation cost of the kernel of the activation function in (8)

A. The ELM Training and Optimization
A simulation training set of mono-exponential decay signals is built based on our experimental parameters as shown in Table I. In this training set, these decay signals are created randomly whose τ is uniformly distributed in the range of 0.5∼3.5 μs and A is uniformly distributed in the range of 0.2 to 2.2 (the SNR is in the range of 50 to 550).
The a.u. in Table I means arbitrary unit, σ τ represents the systemic measurement precision of τ . To evaluate the predictive accuracy of the ELM model, the mean square error (MSE) is introduced. The MSE is defined as where τ i and T i are the predictive value and target value of decay time, respectively, N T is the number of test set. Considering that the typical measurement precision of is τ about ±0.025 μs which is quite small compared with the value range of τ . So we create 30000 decay signals for training. The statistical histogram of true value of τ in the training set is given in Fig. 2. For a range of ±0.025 μs, the amount of training set is about 500.
In our ELM training, the different activation functions are compared at first. The result is illustrated in Fig. 3. We can see that most of the activation functions have the comparable extraction accuracy except for the function of hardlim. We further compare the optimal neurons number of the hidden layer which is noted as P in Table II. Although the activation function of softplus has the smallest P value, the computation cost of  ReLU is quite low. So we adopt the activation function of ReLU in the following simulations and experiments.

B. The ELM Testing and Analyses
After the former analyses, the activation function of ReLU is selected for the ELM testing. The testing set is built according to our typical experiment condition. Our experiment setup is a folded optical feedback cavity ring-down (CRD) system. Its decay time is usually in the range of 0.5 μs to 3.5 μs and its decay signal amplitude is usually the range of 0.2 to 2.2.
A group of 100 decay signals is created and processed by the LM algorithm and the ELM model. The main parameters of these two methods are given in Table III. An example of decay signal is given in Fig. 4. For this signal, only the first 250 data-points are selected out for the ELM  training and prediction. This is because these data-points has better SNR. However, the LM algorithm needs more data-points to extract decay time [11], [12], so the data-points number of the LM algorithm is set to 1000.
The statistical comparison result is given in Fig. 5. It can be seen that the statistical result of the ELM extraction is nearly the same as the simulated true values. However the fitting results of the LM algorithm shows less stability than the ELM extraction. The reason lies in the fact that the LM algorithm needs proper initial values for iteration, or it may fall into wrong local optimization. A conclusion can be made that the ELM method can adapt to our experiment system finely.
The statistical extraction accuracy and precision is given in Fig. 6. It can be seen that the LM algorithm suffers bigger extraction deviation and fluctuation than the ELM method when the cavity decay time is in the range of 3.0∼3.5 μs. For the ELM method, its extraction accuracy and precision are relatively stable at different decay time situations. Actually its accuracy and precision do not decrease too much compared to the LM algorithm. Due to its wide range of adaptability, the ELM method can be greatly practical for the decay time extraction.
Another main requirement is the computation cost. The extraction procedure of the LM algorithm can be given by Here, the sign prime means the corresponding estimated value. The LM algorithm is to achieve the minimum χ 2 by iterations. Although the exponential calculation can be simplified by geometric series expressions [10], large computation cost is still needed in iterations. The number of float point calculation is not easy to estimate, so the computation cost is compared in the same way as Refs. [13], [14], [15], [16] that is averaging the decay signal extraction time.
After simulation analyses, the comprehensive comparisons between the LM algorithm and the ELM method is statistically calculated in Table IV. For the ELM method, the computation cost of decay signal extraction is quite small. What should mention is the time of the ELM training. However, the ELM training only needs a fixed time cost and can be executed offline. For a specific mono-exponential system, the ELM method can obtain the advantage of time consuming over the LM algorithm if the processed signals are enough.

C. Simulation Test
Then the trained ELM model is used to process two groups of simulated decay signals. There are 100 decay signals in each group. The statistical decay time constants are 2.306 ± 0.030 μs and 1.199 ± 0.031 μs, respectively. The corresponding extraction results are shown in Fig. 7.
We can see that the ELM method shows quite fine performance in the decay time extraction. The statistic mean value of the ELM is the same as the true value. The statistic standard deviation is one nanosecond larger than the true value.

D. Experiments
The trained ELM model is further tested by our experimental setup. Here the model is kept the same as former simulations. The setup schematic and a typical decay signal are shown in Fig. 8. This is an optical-feedback CRD system [19]. A continuous wave diode laser (model: RGB Photonics, 1064 nm) is used as the laser source. The laser source is modulated by a 200Hz square wave which is generated by a function generation card (Model M2i.6021, Spectrum). A folded ring-down cavity (RDC) is adopted in this system which is consisted by cavity  are plano-concave mirrors whose curvature radius is about 1m. M 3 is a plane mirror. All the cavity mirrors are coated with high reflectivity films. The typical cavity decay time of this CRD system is in the range of 0.5∼3.5 μs. The transmission light of the RDC is focused on a photodetector (Model APD130C/M, Thorlabs). The detection signal of photodetector is acquired by a data acquisition card (Model M2i.3010, Spectrum). The cavity decay signal is at the falling edge of the square wave.
By misaligning the cavity mirror or changing the cavity length, the measured decay time can be varied [20], [21]. For the lack of known true value of decay time, the results of the ELM model are compared with the LM fitting. The comparison results are listed in Table V, and a typical result is given in Fig. 9.
The extraction performance of the ELM model is little worse comparing with the simulation situations. The possible reason is that the experimental decay signal suffers more noise. We can see from Fig. 8(b) that the experimental decay signal may be affected by some systemic electromagnetic interference, which doubtlessly changes the statistical characteristics of the background noise. Furthermore, study has implied that the experimental decay signal was also combined with the Poisson noise [10]. All these factors were not trained in our model. However,  the ELM method still achieves almost the same extraction result with the LM algorithm. The fluctuation of relative errors is all less than ±2 nanosecond.

IV. CONCLUSION
This paper presents an extreme learning machine for the mono-exponential decay time extraction. The extraction procedure of the ELM is derived. Its theoretical precision, accuracy, and the computation cost are analyzed preliminarily. The ELM model is trained by simulated decay signals. Then it is tested by simulation and experiment, respectively. The ELM model is simultaneously compared with the LM algorithm which is deemed as the defacto standard of mono-exponential decay extraction. Both the simulations and experiments show that the ELM method has quite fine extraction performance. The fluctuation of relative errors of experimental signal extraction is less than ±2 nanosecond when decay time is in the range of 0.9 μs∼2.2 μs. Compared to the traditional LM algorithm, the ELM method does not need to set the initial value and iteration. Therefore, it can obtain the advantage in computation cost if the number of processed signals is large enough.
The current ELM model for the mono-exponential decay time extraction is inevitably affected by noise, so we think some noise preprocessing methods may be helpful for its extraction performance such as the neighborhood smoothing [12] or weighting [22], [23] methods. Furthermore, we think the deep reinforcement learning methods can also be used for the extraction of mono-exponential decay time, which may improve the extraction performance further.