Intelligent Signal Detection Under Spatially Correlated Noise

Detecting the signal of the antenna array is a major problem in theoretical research and practical application. In this paper, several new methods are given for the number of signals detection at first, secondly, a new method called Principal Component Analysis of Signal Estimation (PCASE) will be introduced which can simultaneously detect the number of signals and the direction of arrival. In recent decades, the signal detection method based on the information theory criterion has been widely studied. The problem has been adequately solved under the assumption of uncorrelated white noise. However, considering the actual situation of wireless communication, the noise is spatially correlated or the noise information is unknown. In this case, traditional methods such as Akaike’s information criteria (AIC), minimum descriptive length (MDL) and sparse and parameter approach (SPA) will often lead to a wrong estimation. Therefore, this paper introduces an improved eigenvalue correction method for the number of signals, and applies it to two new methods: the improved eigenvalue gradient method (Im-EGM) and the improved eigen-increment stop rule (Im-EISR), and studies a new estimation algorithm based on signal cancellation (SC). In addition, previous algorithms for estimating the direction of arrival (such as MUSIC, ESPRIT) require the number of known signals to estimate the direction of arrival. Therefore, this paper proposes a new method called PCASE, which can estimate the number of signals and the direction of arrival at the same time. This method combines the SPA and the Principal Component Analysis method (PCA) in machine learning. Compared with the existing methods, the accuracy of these new methods is verified by Monte Carlo simulation.


I. INTRODUCTION
Array signal processing is one of the critical technology for wireless signal processing. The purpose of array signal processing is to obtain some parameters of signal, such as the number of signals, direction of arrival and frequency, by processing the signal received by sensor array. In the research of array signal processing, it is important and realistic to determine the number of signals and signal locations. The direction estimation algorithms widely used in radar, sonar The associate editor coordinating the review of this manuscript and approving it for publication was Ahmed Farouk . and mobile communication, such as MUSIC proposed by Schmidt [1], ESPRIT proposed by Roy and Kailath [2], all need to know the exact number of signals. If the estimated number of signals are not consistent with the actual number of signals, the performance of the above algorithm will decline sharply or even completely fail. Thus, this orientation direction has attracted the attention of many researchers in recent decades [3]- [16].
Eigenvalues and eigenvectors of the observed covariance matrix provides an effective method in multiple applications, such as pattern classification, econometrics, statistical inference and signal processing [3]. The well known VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ methods of Akaike's Information Criterion (AIC) and Minimum Descriptive Length (MDL) were proposed in [17] and [18], respectively. Both AIC and MDL criteria work perfectly for the scenario of uncorrelated signal/noise and use the same computational function but with different penalty terms. To improve the AIC and MDL criteria, a new statistical approach was introduced by Wax and Kailath in [19], which does not require any subjective threshold settings and the estimation of signals number is determined by minimization of AIC and MDL criterion. Thereafter, the traditional AIC and MDL criteria have been analyzed and improved in [13], [20]. However, most of the frequency spectrum of noise is mainly non-white low frequency spectrum, the white Gaussian noise passing through the channel is affected by the channel frequency and becomes colored, that is, the noise with uneven power spectral density function, called colored noise, which is correlated noise. The traditional criteria for estimating the number of signals are based on information theoretic criteria and perform well under the assumption of white Gaussian noise, but they always fail to detect the correct number of signals when the noise is spatially correlated. It should be noted that when the noise is low-frequency band white noise, it also can be correlated. Solving this problem has triggered enormous research interests in recent years.
Assuming the noise has a band covariance matrix structure, the signal estimation problem was studied in [4] and the spectral matrices were computed by using delayed blocks to eliminate the noise influence in [6]. Considering the correlated noise, new detection approaches based on Gerschgorin radii, called the Gerschgorin disk estimator (GDE), were introduced in [5]. Lu used the Gerschgorin disk estimator to estimate the number of sources for the minimal redundant array in [9]. Furthermore, an improved Gerschgorin disk estimator for source enumeration, which is robust to spatial non-uniform noise, was proposed by [10], and the method of applying the Gerschgorin disk estimator to blind source estimation was proposed by [21]. However, it is not easy to determine a threshold of the radii for the GDE algorithm to separate the signals and noise in practice. Using the analysis of the difference between different eigenvalues, the estimator based on the eigen-increment was introduced in [7], while the eigenvalue gradient methods (EGM) were investigated based on the difference or ratio of different eigenvalues in [8], [11]. The traditional K −means clustering algorithm was applied to estimate the number of signals in colored noise field in [12]. However, based on our tests, most of the methods mentioned above do not provide acceptable accuracy estimation results.
In addition, various methods have been proposed and analyzed for the far-field narrow-band signal location problem [1], [22]- [25], also known as direction of arrival (DOA) estimation, for array observation snapshots in recent decades. Parameter estimation is one of the main methods of DOA estimation. Recently, parameter estimation is carried out by studying the subspace of covariance matrix, which is implemented as a large snapshot of maximum likelihood (ML) method in the case of uncorrelated signal (see [26], [27]). However, when the number of snapshots is small or there is a correlation between signals, it also requires the number of signals, and there is no reliable performance. Moreover, the traditional direction finding algorithms (such as MUSIC and ESPRIT) require an accurate number of direction finding signals to estimate DOA. Therefore, an exact discretization-free method which is a sparse and parametric approach (SPA) proposed for uniform and sparse linear arrays in [28]. In the case of uncorrelated signals, SPA is a large number snapshot implementation of ML estimation, and the number of snapshots is consistent. SPA is suitable for any number of snapshots and is robust to signal dependencies, especially it does not require known user parameters. However, all the above methods are based on the assumption of white Gaussian noise. Therefore, assuming spatial correlation of noise, the introduction of Principal Component Analysis (PCA) method in [29] to deal with correlated noise can well solve the problem. Hence, DOA estimation under noise correlation is a practical problem to be solved in signal processing.
In this paper, the problem of how to reliably detect the number of co-channel signals in the presence of spatially correlated noise is investigated and new algorithms and methods are introduced. The main contributions of this paper are as follows: • Two new approaches, improved Eigenvalue Grads Method (Im-EGM) and improved Eigen-Increment Stopping Rule (Im-EISR) will be introduced based on an improvement of eigenvalue correction. Increasing the difference between the eigenvalues will improve the accuracy of the signal detection algorithm.
• A new estimation algorithm is investigated based on Signal Cancellation (SC), which the robust threshold SC can be set for particular direction and frequency after the short time of training to achieve high estimation accuracy.
• A new method of principal component analysis of signal estimation (PCASE) is proposed, which combines SPA and PCA to solve the problem of spatial correlation noise. Different from other methods, this method can simultaneously detect the number of signals and estimate the DOA. The rest of this paper is structured as follows: in section II, we describe the system model. Several new algorithms for detecting signals are presented, and the validity is verified by numerical experiments in section III. In section IV, we present the PCASE for detecting signal and estimating direction of arrival. Finally, section V concludes this paper.
Notation: In this paper, lowercase boldface letters represent vectors, while uppercase boldface letters represent matrices. A T , A H and A indicate the matrix transpose, conjugate transpose and Hermitian of A, respectively. C m×n and tr(A) are denoted as the vector space of all m × n complex matrices and the trace of a matrix A, respectively. || · || F , j = √ −1 and diag(x) represents the Frobenius form operator, imaginary unit and a diagonal matrix with x as the principal. 201996 VOLUME 8, 2020 Furthermore, A − B 0 means matrix A − B is a positive semidefinite and x 0 means x j ≥ 0 for all j.

II. SYSTEM MODEL
In this section, we introduce two kinds of antenna arrays, the uniform circular array (UCA) and uniform linear array (ULA). UCA is suitable for estimating the number of signals, while ULA is suitable for simultaneously detecting the number of signals and estimate the DOA. First, we give the form of the observation model where Y ∈ C N ×L represents the received signal matrix, the matrix S ∈ C M ×L and E ∈ C N ×L represent the transmitted signal and noise, respectively. L is the number of snapshots, M is the number of signals and N is the number of sensors, respectively. The theoretical covariance matrix is defined as where R ss = E[SS H ] is the signal covariance matrix, which is diagonal in the case of uncorrelated signals. 1 The noise covariance matrix is R nn = E[EE H ]. And denote the sample covariance:

A. CIRCULAR ARRAY
We consider the antenna array system equipped with 8 receive antennas located in a circle with a radius of d meters as shown in Figure 1. The time delay is given by where c is the speed of light, θ and φ are defined in Figure 1, The wave number vector is defined by where ω is temporal angular frequency of the wave, f and λ are the frequency and wavelength of light, respectively. The nth element of steering vector is given by λ [x n cosθcosφ+y n sinθcosφ+z n sinφ] = e −jr T n k(θ,φ) .
The received signal is given by where the columns of matrix A(θ, φ) are the steering vectors. 1 We assume that the signals are in our future work.
As the relation θ ↔ ϑ is one-to-one, so the estimate of ϑ = [ϑ 1 , . . . , ϑ M ] T is equivalent to θ. θ can be obtained by the inverse solution ϑ. If we adjust the spacing of the antennas to always keep the wavelength λ 2 , then ϑ is the frequency parameter.
For an N -element ULA, the mth element of steering vector is given by The array manifold matrix A(ϑ) = [a(ϑ 1 ), . . . , a(ϑ M )] contains the steering vectors. According to [22], [30], a simple phase shift is used to represent the time delay of different sensors, and an observation model is obtained: where t indexes the snapshot. Based on the above assumptions, in order to better estimate the effective parameters. Let where p = [p 1 , p 2 , . . . , p N ] T is the signal power parameter. It can be seen that 0. So is a Toeplitz matrix determined by 2N −1 complex numbers, which can be written as = T (u) for some u ∈ C M ,

III. DETECTING THE NUMBER OF SIGNALS
This section presents several algorithms for detecting the number of signals. VOLUME 8, 2020 The additive white Gaussian noise covariance matrix is R nn = E{n(t) n H (t) } = σ 2 I, with noise variance σ 2 . With the independence assumption and applying traditional AIC and MDL criteria, the number of signals can be estimated which is very important for setting the partition of the eigenvectors of the covariance matrix R yy in MUSIC, ESPRIT, etc.
However, if the noise is correlated, the expression of R yy is rewritten as . . , M are the ordered eigenvalues and eigenvectors of A(θ, φ)R ss A H (θ, φ), σ 2 j and n j , j = 1, . . . , N are the eigenvalues and eigenvectors of .
As the following, two new methods of detecting the number of signals are introduced, which are Eigenvalue Correction Improvement and Signal Cancellation.

A. EIGENVALUE CORRECTION IMPROVEMENT
Considering the existing methods, the differences between the eigenvalues of R yy are not obvious enough for detection, because of the noise correlation. The idea of eigenvalue correction is to try to update the eigenvalues of the covariance matrix R yy to enlarge the difference between the eigenvalues of the signals and noise.
If we can estimate the matrix R yy by considering the off-diagonal elements in and deal with the mixture of signals and noise, the signal number can be estimated directly by using the log-likelihood function F as [19], However, it is very hard to separate information of signal and noise from R yy . Because of that, we provide new applicable approaches and improved methods based on the eigen-decomposition of R yy .
Assuming e i , i = 1, . . . , N , to be the correction value of the eigenvalues and order the eigenvalues of R yy in ascending order as: λ min = l 1 ≤ l 2 ≤ · · · ≤ l N = λ max . On the one hand, considering physical meaning of correction value, we need to add white noise to arrays, such that and on the other hand, in order to smoothen colored noise effectively, Based on the idea of eigenvalues correction, we define a new correction method as followings: • Order the eigenvalues in ascending order: l 1 ≤ l 2 ≤ · · · ≤ l N . • Define the correction factor as • Update the corrected eigenvalues as • Rearrange c i in descending order: According to this correction, the difference between eigenvalues of the signals and noise can be enlarged and it will contribute to improve the estimators. This is because, by performing the eigenvalues correction as above, large value corrections are added to enlarge the bigger eigenvalues (normally they represent signals) and small value corrections are put to the smaller eigenvalues. Increased difference between eigenvalues will improve the accuracy of signal detection algorithms. Two algorithms based on the corrected eigenvalues are introduced in the following.

1) IMPROVED EIGENVALUE GRADS METHOD (IM-EGM)
Considering the traditional eigenvalue grads method based on the eigenvalues correction mentioned above, the number of signals can be estimated as follows: Firstly, we calculate the average grads of all corrected eigenvalues as Secondly, each grad between adjacent eigenvalues is calculated as Finally, comparing c i with c from i = 1 to N − 1. If c i ≤ c, the algorithm finishes and the estimate number of signals is equal to i − 1. The computational complexity of the core step of the method is O(N 3 + N 2 + 7N ).

2) IMPROVED EIGEN-INCREMENT STOPPING RULE
In this subsection, a stopping rule is provided for the grads of eigenvalues to improve the accuracy of estimation. A single eigen-increment stopping rule (EISR) is defined as where C EISR is the estimation of the signal power and is given by and f (N , L) is a coefficient function with the following features [7]: Using the EISR, the detection criterion can be described as follows: if (24) it stops and the estimate of the number of signals is i. The computational complexity of the core step of this method is The observed covariance matrix R yy includes the mixture information of signals and spatially correlated noise. The idea of signal cancellation is to subtract the signals based on their eigenvalues and eigenvectors which are larger than the eigenvalues and eigenvectors of noise, and stop at particular step when only the noise is left.
Obviously, after subtracting all the signals, the covariance matrix R yy should only have the noise, and this can be measured by comparing the Frobenius norm of covariance matrix at each step.
Assuming there are d signals, the algorithm can be expressed as where u j , j = 1, 2, . . . , d, d + 1, d + 2, . . . , N are the eigenvectors of R yy according to the eigenvalues l 1 ≥ l 2 ≥ · · · ≥ l N . It should be noticed that when j > d, we can derive the covariance matrix R j yy by subtracting only one item of noise. Furthermore, considering the practical properties of noise, the difference of the generalized Frobenius norm of R d+1 yy and R d+2 yy is very small. To cope with the wide value range of the real data, we normalize their Frobenius norms by Assuming that the stopping threshold is SC, the algorithm will stop when Then the estimated number of signals is j − 1.
Another advantage of signal cancellation method is that the robust threshold SC 2 can be set for particular direction and frequency after short time of training to achieve high estimation accuracy.

C. NUMERICAL RESULTS
In this section, first, simulation results are presented by showing the successful detection probability of new estimation methods. The system is simulated as shown in Figure 1. Eight antennas are uniformly located on the circle with radius 3.5 2 The threshold SC can be set by the experience based on different scenarios. m. In all simulations, we assume that there are two independent signal sources impinging from 45 o and 90 o respectively, but the noise is spatial correlated. Parameter Settings are shown in Table 1. We assume that the correlation matrix of noise is a Symmetric Toeplitz matrices (or called as centrosymmetric matrix), and the correlation matrix element is α |i−j| , i, j = 1, . . . , N . The correlation of noise is generated by correlation matrix and controlled by the parameter α, smaller value of α means less correlation between the noise and the correlation is stronger when α increases. The optimal parameter settings are shown in Table 2. After determining the two values, the value with the probability reaching 1 is selected according to the probability of successfully estimating the number of signals. The best value is greater than or equal to this value.   Figure 2 presents the success estimation probability of EGM algorithm, where EGM denotes the estimation based on original eigenvalues, and Im-EGM denotes the corrected eigenvalues based estimator. The noise correlation parameter α varies from 0.5 to 1. For the spatial correlated noise scenario, EGM works well when SNR is greater than 0 dB, while there is no obvious difference that can be seen between EGM and Im-EGM. Generally speaking, the Im-EGM performs similarly to EGM.  The estimation algorithm based on EISR is shown in Figure 3, where the estimator based on original eigenvalues is denoted as EISR and Im-EISR represents the estimation using corrected eigenvalues. The correlation between noise is varying from weak α = 0.5 to strong α = 1. The estimator based on EISR can provide accurate detection when SNR is greater than 5 dB and noise correlation is weak, while it requires high SNR to provide acceptable detection results when the noise correlation increases. It worth to notice that Im-EISR works better than EISR when there are weak spatial correlations between noises. Simulation results of success estimation probability of signal detection by applying the signal cancellation algorithm is illustrated in Figure 4. First of all, comparing to EGM, Im-EGM, EISR and Im-EISR, the simulation results show that the estimation algorithm based on signal cancellation works better in low SNR regime, but the success estimation probability increases slowly with SNR. The simulation results of new estimation algorithms are compared with each other and existing well known methods GDE [10] and K −means [12] in Figure 5. The noise correlation parameter is α = 1 for all the curves. It can be seen that the estimation algorithm based on signal cancellation performs the best in low SNR regime, but the successful probability of GDE method increases more quickly when SNR increases. The K −means algorithm and estimation based on Im-EISR are worse than the others. Comparing to the original criteria, effective methods have been found in the correlated noise environment and they are verified by the simulation.

IV. PCASE
The methods shown in section III can only improve the accuracy of signal number estimate, in this section, the PCASE algorithm will be introduced simultaneously to detect the number of signal sources and estimate DOA. Based on the popular machine learning method PCA [29] and the basic idea of SPA (see in [28]), we study the new estimation method PCASE. The flow chart is shown in Figure 6, which is described in detail below. Remark 1: In this paper, we assume that rough information of the noise is known, which mean it is not necessary to know the exact information of noise. 3 Herein, we apply the pre-whitening technique by using the information of noise at different times slot, but adjacent frequencies and bandwidths. At least, it is applicable and could improve the detection accuracy, which is verified by the simulations.

A. PRINCIPAL COMPONENT ANALYSIS (PCA)
Principal component analysis (PCA) is a technology widely used in dimensionality reduction, lossy data compression, feature extraction and data visualization, and is an unsupervised linear method. PCA uses orthogonal transformation to transform a group of variables which may be correlated into a group of linearly uncorrelated variables. If the noise is correlated, the covariance matrix of the noise is not a diagonal matrix. Therefore, in order to obtain independent noise, PCA was used to reduce the noise dimension.
The spatially and temporarily white Gaussian noise covariance matrix is R nn = E{n(t 1 )n H (t 2 )} = diag(σ )δ t 1 ,t 2 , with noise variance parameter is σ = [σ 1 , . . . , σ N ] T ∈ R M + and δ t1,t2 is a delta function that equals 1 if t 1 = t 2 or 0 otherwise. And the date snapshots are uncorrelated with each other. Now, if the noises are correlated, R nn does not have this form. In order for R nn to have this form, let's do the following for noise E: • Centralize all samples of E: e(t n ) ← e(t n ) − e; • Calculate the noise covariance matrix EE H ; • Eigenvalue decomposition of the covariance matrix EE H : • Take the eigenvalue vector corresponding to the maximum K eigenvector w 1 , w 2 , . . . , w K . The final form is where w i is the eigenvector corresponding to the largest K eigenvalues of S 0 = 1 L L n=1 (e(t n ) − e)(e(t n ) − e) T , e = 1 L L n=1 e(t n ). B is the coefficient matrix, and its covariance matrix should be the diagonal matrix, which can be set as diag(σ 0 ), σ 0 = [σ 01 , . . . , σ 0K ] T ∈ R K + . The expression of R nn is rewritten as In this way, the number of parameters does not increase when solving SDP problems in the next subsection. SDP was explained in the next page.

B. SPARSE AND PARAMETRIC APPROACH (SPA)
Based on the established covariance fitting criterion and convex optimization, SPA performs parameter estimation in 3 The exact information of noise means we know all the information of noise at the time of doing detection. The rough information of noise means we only have to know the information of noise around detection frequency at the time before the detection the continuous range. It is a sparse parameter method without discretization. Unlike existing parameterization methods, SPA method can detect the number of signals and estimate the DOA. The method consists of the following parts.

1) SEMI-DEFINTE PROGRAMMING (SDP)
When both R yy and R yy are invertible, in order to estimate the unknown parameters, we consider the following covariance fitting criterion (see [23], [31], [32]): where R −1 yy is present with the noise and σ 0j > 0 for j = 1, 2, . . . , N , and f is the distance between R yy and R yy .
Remark 2: The covariance fitting criterion makes use of the assumption that the information signal is not correlated, thus obtaining the expression of R yy in (2). However, the theoretical explanation proposed in [23], [31] proves that the criterion is robust to the correlation of signals. Therefore, the proposed method maintains this robustness under the same criterion.
It can be derived with some simple algebraic manipulations that According to equation (12), the structure of R yy under constraint conditions T (u) 0 and σ 0 0 is obtained as Under the condition of semi-positive definite T (u), the distance f is expected to be the minimum, and the problem can be converted into the following optimization: The number of 2N has been expressed in R yy , which will lead to the redundancy of R yy . However, the redundancy problem will not affect the currently estimated R yy . The handling of this problem is described in the next subsection. So the above optimization problem can be written as min u,σ 0 0 After a series of derivations (see in [28]), the above optimization problem is finally converted into the following SDP problem min X,u,σ 0 0 tr(X) + tr( R −1 yy R yy ) where X R yy . VOLUME 8, 2020 Therefore, the problem in (33) can be expressed as an SDP problem and thus is convex, so the SDP problems can be iteratively solved using the standard CVX toolbox, a Matlab package for specifying and solving convex programs ( [33], [34]). We can get the estimated value R yy = T (u * ) + Wdiag(σ * 0 )W H of R yy , where u * and σ * 0 are the solutions of the SDP problem.

2) POSTPROCESSING
The signal and noise in the estimated covariance matrix can be separated by the post-processing method, so that the signal part can be represented by as few signals as possible on the basis of the minimum description length principle. Since there is redundancy on the diagonal of R yy , and the effect of redundancy is double, the solution of SDP problem cannot be guaranteed to be unique, so it cannot be directly used as the final estimation of noise variance. Therefore, the solution needs to be processed by using post-processing.
R yy is expressed as follows where T ( u) = A( θ)diag( p)A H ( θ) 0 and σ 0 0 are the estimate of and nosie covariance. The u and σ are satisfing that this is one form of decomposition, but it actually lists all the possible forms. Based on T ( u) = T (u * ) − δI ≥ 0, and δ = λ min (T (u * )) is the smallest eigenvalue of T (u * ), so the decomposition is unique. In this way, the solution of SDP is unique. In this condition, it is possible to conclude that the final (u, σ 0 ) estimates have good statistical properties.

3) SOLVING ϑ AND p
The next step is to estimate parameters ϑ and p for given T ( u), which is based on the classical Vandermonde decomposition for semi-definite Toeplitz matrices ( [28], [35], [36]). According to Vandermonde decomposition, ϑ and p can be uniquely determined by T ( u). In practice, ϑ and p can be obtained by the following method. Given T ( u) = A( ϑ)diga( p)A H ( ϑ). It's easy to prove that where p 0, and A 2,...,M ( ϑ) just takes the first row of the matrix A ϑ . According to [37], Prony' s method can be used to solve ϑ and p in this kind of equations. For details of the process, please refer to [37].

C. SIMULATION RESULTS
In this section, we illustrate the performance of the proposed PCASE method. In simulation, we consider M = 2 uncorrelated signals with power p • = [3, 1] T and frequency vector ϑ • = [0.10, 0.50] T . Each signal is generated with constant amplitude and random phase, according to the general settings in [31]. The SNR defined as 10log 10 Parameter Settings are shown in Table 3. In the first case, we consider an array with 8 sensors. Figure 7 shows the PCASE algorithm's probability of success estimation. 4 The noise correlation parameter α ranges from 0.5 to 0.9. It can be seen from the figure that PCASE can provide acceptable test results when the SNR is less than 0 dB, regardless of the strong or weak noise correlation. When the SNR is greater than 5 dB, 100% accurate detection results can be provided. As shown in Figure 8, we compared the simulation results of three algorithms: the PCASE, Im-EISR, SC, GDE and the existing well known methods K-means. The noise correlation parameter of all curves is α = 0.5. It is easy to see that in the case of low SNR, the estimation algorithm based on PCASE has the best estimation effect, but the PCASE method's probability of success grows slowly when SNR increase. However, PCASE can not only detect the number of signal sources, but also estimate the Direction of arrival. Therefore, PCASE is effective in detecting the number of signals.
In following, we show that PCASE can detect the number of signal sources and estimate DOA (θ m = arcsin(2ϑ m − 1)) at the same time. Considering M = 3 and three signals have p • = [3, 2, 1] T and ϑ • = [0.10, 0.15, 0.50] T . In Figure 9, when N = 8 and α = 0.5, the simulation result shows that PCASE is more effective and accurate than SPA under correlated noise.  Considering large scale antenna array, we increased the number of receiving antennas and the noise correlation coefficient, which a ULA with N = 30 is used to receive the signals and α = 0.9. The simulation result is shown in Figure 10. The SPA approach does not work well in this case. However, our method can accurately detect the number of signals and give their directions. In addition, We have regressed our experimental results with a function (y = a * x b ), which is able to classify the signal and noise into two distinct categories. Therefore, the validity of our method is verified.
According to the linear regression curve in Figure 10, the boundary value and the middle of this curve are selected as thresholds to judge the number of signals. Figure 11 shows false alarm rates under different thresholds. As shown in the figure, false alarm rate will increase with the decrease of threshold value. When we select the appropriate threshold value, false alarm rate will be reduced.

V. CONCLUSION
In this paper, the problem of detecting signals with correlated noise has been investigated. The traditional criteria of AIC and MDL were reviewed and the reason of their failure in correlated noise environment was discussed. Furthermore, new criteria were introduced, such as Im-EGM, Im-EISR and SC. Im-EGM and Im-EISR can improve the accuracy of the signal detection algorithm by increasing the difference between the eigenvalues. SC can set a robust threshold for a specific direction and frequency when the training time is short, and achieve high estimation accuracy. The simulations show that all the methods perform well when SNR is greater than 5 dB. At low SNR regime, the estimation based on SC works better than the others. For future work, the SC method will be improved based on robust threshold setting. In addition, we also propose a method that can simultaneously detect the number of signals and estimate DOA. This method combines machine learning and sparse parameter method. Simulation results show that this method can not only detect the number of signal sources accurately, but also estimate the DOA of signal sources. Compared with other methods for detecting the number of signals, this method is effective in the case of low SNR. Therefore, the new method is a powerful tool for detection. We will be considered to extend our work and try to derive effective methods to study coherent signal estimation in our future work.
JOHN THOMPSON is currently a Professor of signal processing and communications with the School of Engineering, The University of Edinburgh. He specializes in antenna array processing, cooperative communications systems, and energy efficient wireless communications. He has published in excess of 300 articles on these topics, including one hundred journal article publications. He is also the Project Coordinator for the EU Marie Curie International Training Network Project ADVANTAGE, which studies how communications and power engineering can provide future smart grid systems. He was an elected Member-at-