Performance Assessment of Transient Signal Detection Methods and Superiority of Energy Criterion (EC) Method

Radio frequency fingerprinting (RFF) based on RF transients is one of the most effective techniques for improving wireless security. For an efficient RFF development, RF transients need to be accurately detected. However, the detection of the transient starting point remains a main challenge due to the channel noise. In the literature, several methods have been presented to detect the starting point of the transient signals. As an alternative to these methods, this study proposes a method that utilizes Energy Criterion (EC) technique for the first time. In order to test its performance, firstly, an extensive dataset consisting of Wi-Fi signals recorded under realistic Signal-to-Noise Ratio (SNR) conditions is created. Using the provided dataset, the proposed method as well as common transient detection methods are employed for transient start detection. Then, the effect of SNR on the performance of transient start detection is evaluated. Moreover, a performance comparison between the methods is provided based on their respective computational speed and complexity. The results prove the feasibility and efficiency of the proposed method to detect the transient starting point for RFF of Wi-Fi device identification. As to the knowledge of the authors, this study is the first report that comparatively assesses the transient detection methods by using extensive data under realistic noise conditions.


I. INTRODUCTION
Today, the growing usage of modern wireless communication technologies introduces users to various external security threats. Users might be confronted with undesired consequences due to malicious attacks. To prevent such attacks, the techniques based on physical-layer identification of wireless communication devices, which is also referred to as Radio Frequency Fingerprinting (RFF), can be used [1]. In RFF process, typically, distinctive features (RF fingerprints) of physical waveforms transmitted from a wireless device are utilized to classify authorized users in the network. These features are then used to identify the threats. Here, either the transient or steady-state regions of the transmitted signals can be used to extract the features. It is important to note that most of the robust and subtle features can be extracted from transient regions [2]- [4]. Therefore, it is The associate editor coordinating the review of this manuscript and approving it for publication was Qinghua Guo . necessary to accurately detect transient signals which constitutes a key stage of an efficient RFF development. However, detecting of transient signals is not an easy task due to its shorter duration and channel noise. Thus, there are some inevitable difficulties that have to be overcome in transient detection. Among them, providing an accurate and consistent way of transient starting point detection remains an important challenge [5].
In the literature, several methods that accurately detect the starting point of the transient signals have been proposed so far. One of them is Variance Fractal Dimension Threshold Detection (VFDTD) which detects transient signal by using the fractal dimension calculated from the variance of signal amplitude [6]. Bayesian Step Change Detection (BSCD) method is another method that detects the transients by means of a posterior probability distribution function [7]. Phase Detection (PD) is the other method that exploits phase characteristics of signals for transient detection [8]. Mean Change Point Detection (MCPD) method is also presented in [9] to detect the transient starting point by calculating the maximum of the difference of statistic. In [10], step change detection scheme presented in [7] is modified by using Bayesian ramp change detector. Further, a simple and accurate transient detection method using self-adaptive threshold based on the MCPD-based energy trajectory is proposed in [11].
In this study, as an alternative to existing transient detection methods, a novel method that utilizes the Energy Criterion (EC) technique is proposed. Conventionally, the EC is applied to detect the arrival times of the electrical signals as a typical signal pre-processing step for the acoustic partial discharge location [12], [13]. It is based on the assumption that the arrival of a signal is characterized by a variation of its energy content. To the best of our knowledge, there has been no study on the performance of EC in the transient signal detection of wireless devices. In this context, this study also aims at evaluating the performance of the proposed method under realistic Signal-to-Noise Ratio (SNR) conditions. To that end, initially, an extensive dataset consisting Wi-Fi signals captured from a Wi-Fi device is created. Then, different levels of noise (−3 to 25 dB) captured during data collection are added to the dataset. Next, the effect of additive noise on the transient start detection performance of the proposed and well-known methods [6]- [9] is experimentally demonstrated. Besides, the detection performances of the methods are evaluated by comparing their respective computational complexity and CPU elapsed time. The results show the efficiency of the EC-based transient detection method in comparison with the existing methods.
The contributions of this study can be summarized as follows: (a) This is the first study that utilizes EC technique to detect transient starting point for RFF. (b) Transient starting point detection performance of existing methods is comparatively assessed by using extensive data under realistic noise conditions for the first time. The rest of the paper is structured as follows. Data acquisition system is described in the following section. Then, the existing transient detection methods are overviewed in Section III. Next, the details of EC-based transient detection method are presented in Section IV. Further, Section V provides the comparative assessment while conclusions are drawn in Section VI.

II. DATA ACQUISITION AND PREPROCESSING
Typically, a transient-based RFF method consists three main stages, namely data acquisition, signal processing, and classification. In signal processing stage, transient regions of the transmitted signals are used to extract the distinctive features. In classification stage, the extracted features are utilized for classifying the transmitting devices. Among these stages, data acquisition stage has a considerable importance for both in transient detection and accordingly RFF implementation. Even a small mistake or drawback might adversely affect the other stages. This then results in inaccurate transient detection and hence poor device identification ability. Particularly, the receivers (high-or low-end) commonly used in data acquisition bring some drawbacks [14]. The most important one is higher sampling rate that leads to increase the data size. These types of receivers also generate undesired frequency components (spur signals). To alleviate these drawbacks, it is necessary to preprocess the data properly. In this section, after describing the data acquisition system used in this study, the steps followed to preprocess the data is explained briefly.

A. DATA ACQUISITION SYSTEM
To perform Wi-Fi signal acquisition, the system shown in Fig. 1 was used. As shown in the figure, Wi-Fi signals were captured from a Wi-Fi device (smartphone) in a laboratory environment. The laboratory was isolated in the second underground floor of a nine-story building. During the data capture process, redundant electronic devices in the environment were switched off. Wi-Fi signals were directly captured through a high-end receiver (Tektronix TDS7404 oscilloscope / 20 GSPS) without any down conversion to ensure that impairments on distinctive transient characteristics are minimized. To capture Wi-Fi signals, a commercial Wi-Fi antenna was connected to the oscilloscope. The distance between the smartphone and the antenna was 30 cm. Moreover, the recorded data was transferred to a computer. It should be noted that the smartphone was switched to flight mode during the recordings so that the undesired signals generated from the smartphone can be eliminated. Thus, 150 Wi-Fi signals were captured. As shown in Fig. 2, a typical record includes three main parts: noise (channel noise), transient and steady state.

B. PREPROCESSING
As discussed at the beginning of this section, the oscilloscope (high-end receiver) used in data acquisition system demands the extended memory because of the higher sampling rate used for recording signals. This in turn results in increased data size and computational cost. In addition to these drawbacks, the oscilloscope generates undesired frequency components (spur signals). Hence, in order to improve the performance of RFF implementation as well as transient starting point detection, the captured intermediate frequency (IF) signal is needed to be transformed into analytical signal by using Hilbert Transform (HT) [15]. This is based on the fact that analytical signals are generally useful to measure instantaneous characteristics of a signal such as the instantaneous amplitude, phase, or frequency. It is worth noting that these characteristics can be utilized in the signal processing stage of an RFF method to extract distinctive features. Besides, these characteristics can be used for detecting the transient starting point as described in the following section. In the next step, the analytic IF signal is down-converted to baseband by multiplying with a complex exponential. Finally, Low Pass Filter (LPF), where the cutoff frequency had been set to 90 MHz, is used to suppress the undesired frequency components.

III. EXISTING TRANSIENT DETECTION METHODS
As mentioned previously, the detection of the transient starting point is the main challenge that must be provided accurately. In this context, several methods have been proposed in the current literature. In this section, these methods are briefly presented.

A. VARIANCE FRACTAL DIMENSION THRESHOLD DETECTION (VFDTD)
In this method, the transient starting point is detected by calculating the fractal dimension from variance of signal amplitude [6]. Initially, the fractal dimension D (n) of each signal that is segmented by sliding window is calculated where H is Hurst index which is calculated using least square regression method where and In (3), n i is the sliding window length which should be selected carefully. Indeed, it could be large as long as the calculated dimension is statistically reasonable. Otherwise, the fractality of the signal might be hidden. On the other hand, if it is too small, computational time can be appeared as a drawback.
Moreover, in (4), var {·} is the variance operator, and X (n i , n i ) represents the amplitude changes between consecutive sliding windows and can be determined from In the last step of VFDTD method, threshold (τ ) is required to be set as a mean of fractal dimension of channel noise. Thus, the point n can be determined as the start of transient provided that n and its after z consecutive points are less than the τ .

B. BAYESIAN STEP CHANGE DETECTION (BSCD)
This method is based on an approach that uses Higuchi's method to calculate the variance of fractal dimension for successive portions of the signal [7]. Here, the variance of fractal dimension between two consecutive sequences is proportional to their posteriori probability distribution function (PDF). The maximum value obtained from the PDF is then determined as the start of the transient.
In the first step of this method, subsets of the samples are rearranged as where X (m, k) is subset interval, m is the initial time, and k is the interval time.
The length of the curve, L m (k), for each subset is then calculated by Next, the average value of the k sets (L m (k)) is plotted on a log-log scale. Curve fitting is then performed, and the slope of the curve is considered as the fractal dimension.
In the last step, the start of the transient (m) is detected by following the posteriori PDF where N is the number of samples in the sliding window while d denotes the fractal dimension. VOLUME 8, 2020

C. PHASE DETECTION (PD)
In order to detect the start of transient, PD method utilizes the instantaneous phase characteristics of the analytic signal, (s a (n)) [8]. Typically, s a (n) is expressed as [16] s a (n) = s a I (n) + js a Q (n) where I and Q are in-phase and quadrature components, respectively, s a I (n) is a real-valued discrete signal in time domain (s (n)), and s a Q (n) = H {s (n)} where H {·} denotes the HT. Hence, the signal characteristics such as the instantaneous phase (∅ (n)), the instantaneous amplitude (a (n)), and the instantaneous frequency (f (n)) can be calculated by Then, ∅ (n) calculated in (10) is unwrapped to surpass the phase discontinuities. Hence, to ease the detection process, the absolute value of each element in the unwrapped vector (AV ) is obtained as given in the following Further, a nonoverlapping window of size s is used to calculate the variance of phase for each successive portion of AV . With the help of this process, the variation between the transient and the noise region of the signal is magnified. This can be stored in a temporary vector TV TV (i) = var (AV (d + 1) , . . . , AV (g) ) where i = 1, . . . , N s, g = i × s, and d = g − s. Therefore, the fractal trajectory (FT ) is created by obtaining the difference between the phase variance. Thereby, each element in the FT is compared to the given τ in order to detect the transient start. This process is repeated until the value of the element along with the values of the next four elements satisfy the following condition where τ has been experimentally determined as discussed in [8]. If the above condition is satisfied, then n is considered to be the start of the transient.

D. MEAN CHANGE POINT DETECTION (MCPD)
In MCPD method, the difference between the statistic of samples is magnified, and the position that gives the maximum difference is determined as the start of transient [9]. The temporary vector in (14) is divided into sections: x 1 , x 2 , . . . ,x i−1 , and x i , x i+1 , . . . ,x N where i = 1, . . . , N . To calculate the mean and statistics of each section, the following expression is considered Then, to calculate the average (X ) and statistics (S) of the original sample is calculated by considering whereX represents the average of combined sections. After plotting the curve of S − S i , the maximum point of the curve is determined as the start of the transient.

IV. TRANSIENT STARTING POINT DETECTION USING ENERGY CRITERION (EC)
The EC is well-known technique to locate acoustic and electromagnetic partial discharges. It is commonly used for estimating the arrival time of signals in various applications [12], [13]. The idea underlying the EC is that the arrival of a signal is characterized by a variation of its energy content.
The energy (E i ) of a sampled signal (x) is defined as a cumulative sum of amplitude values [12], [13] where N is the signal length. The signal is separated from the noise part by where δ is a negative trend and can be expressed as Here, δ depends on the total energy of the signal (E N ) and ϑ factor which enables to reduce the delaying effect of δ. Thus, the calculated energy curve (E i ) qualifies a global minimum that is considered as arrival time of signals. In order to exploit EC technique in transient starting point detection, we offer two methods, namely, a (n) characteristics-based EC (EC-a) method, and AV (n) characteristics-based EC (EC-∅) method.

A. EC-a METHOD
In this method, firstly, a (n) characteristics of the analytic signal found in (11) is used to calculate E i . The global minimum on the energy curve is then identified. Hence, the sample that corresponds to the global minimum is determined as the transient starting point. However, there might be several local minimum values within a flat region. In this case, the first local minimum within the region can be selected to determine the transient starting point. It is important to note that the energy curve is highly dependent on the selection of ϑ factor used in (20). Based on the discussion provided in [13], the global minimum of the energy curve is expected to be moved toward the transient starting point when the value of ϑ factor is increased such that ϑ = [1, 2, . . . , 100] under denoised condition. However, when different SNR levels are taken into account, as in this study, the value of ϑ factor needs to be experimentally determined. In this context, it has found that the detection accuracy is significantly improved when ϑ = 30 for the given dataset. As an illustration, Fig. 3 (a)  shows the energy curve obtained by using EC-a for ϑ = 1, 2, 30 while Fig. 3 (b) shows the detected transient starting points.

B. EC-∅ METHOD
The EC-∅ method is based on the use of AV (n) expressed in (13). The idea is to exploit the random change in the noise part of unwrapped instantaneous phase characteristics of the signal in order to establish another random signal which has an approximately equal variance. With the help of this signal, it is expected to obtain the energy curve that monotonically increases at the noise part of the signal. The global maximum point of this curve is then considered to be the transient starting point. Thus, the method begins with taking the absolute differences between each mean window of unwrapped instantaneous phase characteristics of the signal. Then, E i is calculated, and global maximum of the curve is obtained as shown in Fig. 4 (a). Next, the sample which corresponds to the window index providing global maximum of the curve is identified on the unwrapped instantaneous phase characteristics of the signal (Fig. 4 (b)). This also yields the detection of transient starting point as shown in Fig. 4 (c).

V. EXPERIMENTAL RESULTS AND ANALYSIS
In this section, the transient starting point detection performances of VFDTD, BSCD, PD, MCPD, EC-a and EC-∅ methods are experimentally demonstrated. The performance comparison between the methods is provided based on detection accuracy, computational complexity, and CPU elapsed time for detecting the transient starting point. To assess the performance of these methods under realistic noise conditions, different levels of channel noise captured in the measurements were randomly added to the recorded 150 Wi-Fi signals in the range of −3 dB to 25 dB [2].  where p 0 is the estimated start of transient, p is the actual start of transient, and f s is the sampling frequency. Based on the results shown in Fig. 5, BSCD method has low performance at all SNR levels. This can be attributed to the posteriori PDF degradation as discussed in [17]. As shown in Fig. 6., it is possible to achieve better performance when VFDTD method is used. According to the results, only a slight performance degradation is observed at SNR > 15 dB. From the results shown in Fig. 7, similar detection  performance is obtained for MCPD method. PD method has better detection performance at SNR > 10 dB when compared to VFDTD and MCPD method (Fig. 8). Yet, its performance tends to degrade gradually at lower SNR levels. The results achieved for EC-∅ method are presented in Fig. 9. Relatively, it has better performance in comparison with the other methods. However, from the results presented in Fig. 10, it is obvious that EC-a method has superior transient starting point detection performance at all SNR levels. In order to support these findings, the average detection rates of the methods have been calculated. The results are shown in Fig. 11 where MCPD method is the only method that provides the average detection rate below 90%.

B. COMPUTATIONAL COMPLEXITY AND ELAPSED TIME
Computational complexities for the transient starting point detection methods are listed in Table 1. From the table, it is clear that BSCD method has the highest computational complexity (O n 3 ). VFDTD method has also relatively high computational complexity. On the other hand, PD,   MCPD, EC-a, and EC-∅ methods have similar computational complexity (O (n)).
In Table 1, the CPU elapsed times for transient start detection are also presented. Due to the higher computational complexity of BSCD and VFDTD methods, the elapsed time to  detect transient start becomes worse as expected. Specifically, BSCD method exhibits poor detection performance in terms of elapsed time (39.35 sec.). Although PD, MCPD, EC-a, and EC-∅ methods have similar computational complexity, EC-a method provides the lowest elapsed time to detect transient start (0.01 sec). Here, it is worth noting that the resulting elapsed time of EC-∅ method is also efficient (0.03 sec.). As a result, when compared to other detection methods, both EC-a and EC-∅ method outperform all other methods in detecting transient start in terms of computational speed.

C. DISCUSSION
To quickly sum up, BSCD method has poor performance in detection accuracy, computational complexity, and CPU elapsed time. The transient start detection performance of VFDTD method depends on SNR level, and its computational speed and complexity are not optimal. An experimental threshold value is also needed in its implementation. This, however, may bring an important disadvantage in practice. Although MCPD method provides similar detection accuracy with VFDTD method, it has lower computational complexity, and it offers faster computational speed for detection. As an advantage, a threshold is not required to determine in its implementation. However, when compared to MCPD and VFDTD methods, PD method has faster computational speed and better detection accuracy at high SNR levels. It should be noted that unlike the other three methods, PD method utilizes instantaneous phase characteristics of the signal. Still, a threshold needs to be determined in its implementation as such in VFDTD method. On the other hand, both EC-∅ and EC-a methods have same computational complexity with PD and MCPD method. Nevertheless, both EC-∅ and EC-a methods provide considerable performance improvement in both detection accuracy and computational speed. Moreover, as an advantage, a threshold is not required to be set in their implementation. However, the value of ϑ factor highly affects the detection accuracy of EC-a method. For better detection accuracy, it is necessary to select an optimum value of ϑ factor by considering SNR levels. For clean (denoised) data, it is possible to achieve better detection accuracy by selecting the value of ϑ factor as high as possible.

VI. CONCLUSION
In this article, a novel method that utilizes EC technique is developed to detect the starting point of transient signals.
The detection accuracy of the proposed method is evaluated by comparing the results to those obtained from well-known transient detection methods. The results demonstrate that the proposed method is more effective to detect the starting point of Wi-Fi transients at different SNR levels (−3 to 25 dB). It provides an advantage in terms of computational complexity, and it yields superior performance in computational speed over other methods. Furthermore, it is believed that the comparative assessment of the transient detection methods presented in this study could provide an opportunity for researchers to decide optimal transient detection method for RFF of Wi-Fi device identification.