Joint Nuclear Norm and ℓ1–2-Regularization Sparse Channel Estimation for mmWave Massive MIMO Systems

Millimeter-wave massive MIMO can effectively improve the signal-to-noise ratio, but the high-dimensional channel matrix significantly increases the complexity of the classic channel estimation algorithm. On the other hand, millimeter-wave massive MIMO has low rank and sparsity properties in the angle domain. Combining these two properties can effectively improve the channel estimation accuracy. This article proposes a novel millimeter-wave sparse channel estimation method based on joint nuclear norm and <inline-formula> <tex-math notation="LaTeX">$\ell _{1-2}$ </tex-math></inline-formula>-regularization. The basic idea of the proposed algorithm is to formulate the channel estimation problem as a compressed sensing problem. This method constructs an objective function consisted of <inline-formula> <tex-math notation="LaTeX">$\ell _{1-2} $ </tex-math></inline-formula>-regularization, and the resulting nuclear norm minimization problems is optimized via the alternating direction method of multipliers (ADMM) algorithm. The simulation results verified that the proposed method can provide better estimation accuracy compared with the state-of-the-art compressed sensing-based channel estimation methods.


I. INTRODUCTION
Mobile communication systems have been constantly evolving due to new applications and demands. The emergence of the Internet of Everything (IoE) system, which connects millions of people and billions of machines requires the wireless communication systems to have higher transmission rate and lower delay [1]. However, the existing frequency band is no longer sufficient to meet the increasing spectrum demands of wireless communication technology. It is necessary for wireless communication to evolve to a higher frequency band. Millimeter wave (mmWave) communication technology plays a key role in providing more bandwidth, larger capacity, ultra-high data rate and safe transmission [2]- [4].
Due to the weak millimeter wave diffraction ability, it is easily blocked, resulting in a poor wireless transmission. The high path loss in mmWave frequencies can be compensated The associate editor coordinating the review of this manuscript and approving it for publication was Yiming Huo . by massive MIMO technology. The short wavelengths of mmWave enables the placement of dozens to hundreds of antenna elements in an array on a relatively small physical platform. The large antenna array can provide a sufficient beamforming gain. Beamforming can be categorized into analog digital, and hybrid [5]. To reduce the power consumption and the cost, mmWave massive MIMO systems often use a hybrid beamforming structure [6]. The analog part aims to increase the antenna array gain, and the digital part aims to cancel interferences [6]. However, designing appropriate precoding matrix for the system requires a priori known channel state information. Hybrid precoding using channel estimation is still required to improve mmWave system performance.
Nonetheless, millimeter-wave channels have sparse characteristics in the beam space domain. [7]- [9] The application of compressed sensing theory to millimeter-wave channel estimation, using 1 -norm instead of 0 -norm to represent sparsity, transforms the channel estimation problem to the recovery of sparse signals problem, the solution of which can be obtained using compressed sensing algorithm to improve VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ estimation accuracy and to reduce complexity. Compared with classic massive MIMO channel estimation algorithms such as MMSE, LMMSE, LS, etc., millimeter wave channel estimation algorithm based on 1 -norm minimization provides lower complexity, higher performance and stronger robustness. In addition, as the dimensions of massive MIMO increase, compressed sensing algorithms have more obvious low latency overhead, so they have been extensively studied in recent years. In order to further utilize the sparsity of signals, non-convex functions were proposed to recover the sparse signals, such as using q(0 < q < 1) instead of 1 -norm [10], [11]. Although q is more difficult to minimize than 1 -norm, it can reconstruct sparse signals from fewer observation vectors [12]. However, many problems are coherent, and conventional methods such as 1 minimization do not work well. The difference of the 1 and 2 norms, denoted as 1−2 , is shown to have superior performance over 1 method [10]. [11] Using the mixed 2 / 1−2 further expand the range of recoverable sparse signals and improve the reconstruction probability.
In addition to the sparse scattering characteristic, millimeter wave channels also have low-rank properties. In the highdensity urban environment, previous real-world measurement reports indicate that millimeter wave channels propagate in the angular domain in the form of clusters [13]- [16]. This angular expansion leads to the channel matrix having a structured sparse nature, which can be used to improve the estimation performance [17]. At the same time, since millimeter wave channel propagation paths are mostly concentrated in clusters, the correlation of the channel matrix is high, resulting in a low-rank channel matrix, the channel rank of which is much smaller than the sparse level [24]. The low-rank structure of the millimeter wave channel can effectively reduce the channel estimation pilot overhead and the computational complexity. Previous works proposed a method for millimeter wave channel estimation that exploits both the low-rank and sparsity properties of millimeter-wave massive MIMO channel via two-step, which first uses the low-rank property of the channel to recover the received signal, followed by the use of compressed sensing algorithm to estimate the millimeter wave channel gain matrix [18]- [21]. Additional works combined these two characteristics simultaneously for millimeter wave channel estimation at a same time [22]- [24].
In this article, we present a novel multi-objective optimization formulation consist of joint nuclear norm and 1−2regularization for millimeter-wave massive MIMO channel estimation. Moreover, we derived a formulation based on alternating direction multiplication method (ADMM) for optimal solution. The representative simulation results show that the proposed algorithm exhibits faster convergence and improved performance in terms of normalized mean squared error for channel estimation compared with other state-ofthe-art techniques. This article is organized into following sections: Section II introduces the channel model and some basic assumptions used in our study. Section III describes the formulation of the optimization problem and the optimization algorithm is described. Section IV presents and analyzes the simulation results. Finally, the conclusions of this article are given in Section V.
Notation: scalar, vector, matrix are denoted by a, a, A respectively; A T represent the transpose of the matrix, A H indicating conjugate transposition; the actual value and estimated value is represented by A andÂ respectively; the Hadamard matrix product is represented •; () * represents the complex conjugate. the Kronecker product is represented by ⊗; the vectorization of the matrix is represented by vec(A); unvec(A) is the inverse operation of vec(A). I N represents N × N dimensional identity matrix; represents the selection matrix composed of 0 and 1; F is the F−norm of the matrix.

II. SYSTEM MODEL
We adopt a point-to-point downlink wireless communication system model. The BS is equipped with N T transmitting antennas and the UE is equipped with N R receiver antennas. The BS and UE both use uniform liner array (ULA). Considering the hardware overhead of precoding, the structure of hybrid precoding is used at both the transmission and receiver, and F T ∈ C N T is used to denote the precoding matrix, W R ∈ C N R denote the combining matrix. Assuming that each frame of the system is transmitted independently and every frame consists of T blocks dedicated for channel estimation, and the channel state characteristics in each frame remain unchanged, the received signal can be expressed as where y(t) is the observed signal vector after the combiner, m(t) ∈ C N T ×1 is the pilot signal vector. Y = [y(0), · · · , y(T − 1)], M = [m(0), m(2), · · · , m(T − 1)], and n(t) is the additive white Gaussian noise (AWGN) vector. H ∈ C N R ×N T is the narrowed band millimeter-wave channel matrix. By nature, millimeter waves are easily absorbed by surrounding objects, bringing sparse characteristics to the millimeter wave channel. Assuming that there are L paths between the base station and the user, the millimeter wave channel is expressed as follows where ρ denote the average path loss, α l is the l-th path complex-valued gain, α is a sparse vector which the number of non-zero elements in α is equal to the number of paths L.
The array response vector of transmitter and receiver, a T (φ l ) and a T (θ l ), are given by where λ is the wavelength and d is the antenna element spacing, θ l , φ l ∈ [0, 2π ] are the angle of arrival (AoA) and departure (AoD) which are generated according to the Laplace distribution. In order to make use of the sparse characteristic of millimeter-wave channel, an alternative representation for channel is based on the beamspace model that is defined as where S ∈ C N R ×N T represents the channel gain of H, A T ∈ C N T ×N T and A R ∈ C N R ×N R are the steering vectors generated based on Discrete Fourier Transformation (DFT) that is defined as Without loss of generality, we assume that the pilot signal in each block is a unit vector with constant power, i.e. M ∈ I N T ×T . Let A = F T M and substitute equation (5) into equation (1) and vectorize it where s contains the arrival angle θ l and departure angle φ l and their path gain information α l , l ∈ [0, 1, · · · , L − 1]. According to compressed sensing theory, the channel estimation problem is transformed into a signal recovery problem where ε is the error threshold. Wen et al proved that when the sensing matrix satisfies the RIP, high-dimensional signals S can be recovered from low-dimensional signals y to achieve millimeter wave channel estimation [12] ( where δ ∈ (0, 1), write (10) into an unconstrained form to obtain 1 −regularized form: where λ > 0 is the weighted factor, and question (12) is a LASSO question which has been studied extensively, and easily solved by fast iterative shrinkage-thresholding algorithm (FISTA).

III. PROPOSED JOINT SPARSE CHANNEL ESTIMATON ALGORITHM A. OPTIMIZATION FORMULATION
Before elaborating on the proposed algorithm, we perform low-rank matrix sampling preprocessing on the observation signal of formula (1). We adopt the receiving structure proposed by Vlachos et al that randomly sample the signal passing through the combiner, and output only part of the observation signal [23].
where Y is the sub-sampling matrix of Y. If sub-sampling noise is ignored, their (i, j) elements are all equal, indicating the set of observed element positions in the matrix Y, | | = T . Observed signals Y can be recovered using matrix completion theory, such as Singular Value Threshold- Reference [25] has proved that when the sub-sampling rate satisfies equation (15), the probability of recovering the original signal without error 1 − c n 3 if the rank of the matrix to be restored is R ≤ n 1/5 , when the sub-sampling rate p satisfies equation (16), the probability of recovering the original signal without error is at least where n = max(N R , N T ), C and c are both mathematical statistical constants. Therefore, millimeter-wave channel estimation algorithm based on low-rank matrix completion requires several observations of O(R max(N R , N T ) 5/4 ). Because the sparse characteristics of the millimeter wave channel is ignored, only suboptimal sampling complexity can be achieved. The combined sparse characteristics and low rank structure can effectively reduce the sampling complexity [18]. Inspired by Vlachos et al. [22], combining the 1−2 minimization and nuclear norm minimization problems, a novel multi-objective optimization problem is proposed where the nuclear norm of Y in the objective function imposes its low rank property. whereas 1−2 of s enforces its sparse structure. τ Y > 0, τ S > 0 are the weighting factors which depend in general on the number L of the millimeter-wave MIMO channel propagation path P () represent subsampling operator

B. THE PROPOSED JOINT CHANNEL ESTIMATION ALGORITHM
Formula (17) is a multi-objective optimization problem. In order to obtain the global optimal solution, we derive an algorithm based on ADMM. A similar approach has been adopted in by Vlachos et al. [22], however the method is quite different. Firstly, our algorithm is based on compressed VOLUME 8, 2020 sensing, secondly, we creatively uses 1−2 -regularization to enforces sparse structure in angle domain, finally, we proposed new step to solve 1−2 -regularization problem. Equation (17) in unconstrained form can be written as However, it is very difficult to solve 1−2 . Inspired by the [10], We can use the iterative method to solve equation (19) where (20), we resort to an ADMM algorithm and reformulate it as follows min Y,s,X,z where x = vec(X). Establish the augmented Lagrangian function according to equation (21) L 1 (Y, s, z, X, v 1 , where v 1 ∈ C N T N R ×1 , V 2 ∈ C N R ×T represent the Lagrange multipliers, ρ represents the algorithm iteration step size. According to ADMM approach, at the k-th algorithmic iteration with k = 0, 1, . . . , I max the following separate subproblems need to be solved 2 ) (25) Next, we will show the solving steps of questions (23) to (26) in detail. During the k-th iteration, the solution of the variable Y (k+1) can be equivalent to the following subproblem Note that the last item in (29) has no effect on the update of the variable Y (k+1) , and ignore this item There have been a lot of studies on the solution of problem (30), such as ADMM or SVT, and we adopt SVT algorithm to solve it where U (k) ∈ C N R ×R , T (k) ∈ C N R ×R are the left and right singular vectors, respectively, of the matrix 2 . The solution of the variable s (k+1) can be equivalent to the following sub-problem similar to the update process of variable Y (k+1) , equation (32) can be rewritten into the following form rewrite equation (33) into a new sub-augmented Lagrange function differentiating with respect to s and making it equal to 0 to get the updated result of variable s (k+1) where e = (z (k) −1/ρv (k) 1 ). The solution of the variable z (k+1) can be equivalent to the following subproblems 155412 VOLUME 8, 2020 equation (36) can be regarded as a 1 -regularization problem, which is a standard LASSO problem, the update equation is as follows , sign() is sign function, Re() is the real part of the signal, Im() is the imaginary part of the signal.
Finally, the solution of the variable X (k+1) , we can firstly update x (k+1) and then recover X (k+1) using X (k+1) = unvec(x (k+1) ) differentiating with respect to x and making it equal to 0 to get the updated result of variable X (k+1) 2 ) + ρy (k+1) ), and where I R is the identity matrix with dimension N R × N R . Through the above discussion, the millimeter wave channel estimation problem is transformed into a joint minimization problem of nuclear norm and 1−2 -regularization. In order to solve this problem, a multi-objective optimization channel estimation algorithm based on ADMM algorithm is proposed in this article. And the specific flow of the algorithm is described in Algorithm 1.
In (19), the first term is used to constrain the low-rank characteristics of the received signal, the second term is used to constrain the sparse characteristics of the millimeter wave channel, the third term is used to constrain the sub-sampling error, and the fourth term is used to constrain the fitting error. In order to balance the nuclear norm minimization problem and the 1−2 -regularization problem, a weighting factor τ Y , τ S is proposed. the weight factor τ Y , τ S is set according to the following formula [23].
During the iteration process, the value of the channel gain matrix S will gradually approach the true value, and the number of iterations will affect the final millimeter wave channel estimation accuracy. Therefore, by setting an appropriate number of iterations, a high-accuracy channel estimation result can be obtained by Algorithm 1. The simulation results will be shown in the next section.
The total complexity of the algorithm is mainly affected by Step 3. It is to use the matrix completion theory to recover the signal matrix from the observed sub-sampled signal. The complexity increases with the dimension of the Algorithm 1 Jointly Nuclear Norm and 1−2 -Regularization Based mmWave Channel Estimation Input: received matrix Y , sensing matrix , weight parameters τ Y , τ S , ρ, iteration times I max output: H 1: Initialization At the same time, the estimation accuracy of Algorithm 1 is also affected by the sub-sampling rate p, which can effectively reduce the sampling complexity of the system. However, if the sub-sampling rate is set too low, Step 3 in Algorithm 1 cannot accurately recover the signal matrix, which affects the algorithm the overall performance. The experimental results in [26] verify that the sub-sampling rate p = 0.5 can effectively recover the millimeter wave channel.
Only the narrowband systems are considered in this article and, and our work can expand to broadband systems. In broadband systems, the number of antennas or the transmit bandwidth increase significantly, the propagation delay is different among antennas. This effect makes the array response vary with frequency, causing the beams to deviate in OFDM systems [27]. Hence, in broadband systems, the beam squint effect is a factor that has to be considered.

IV. SIMULATION RESULTS
In order to verify the reliability and stability of joint nuclear norm and 1−2 -regularization channel estimation algorithm proposed, we conducted simulation experiments. The Monte-Carlo method is used to verify the performance of the proposed algorithm under different system parameter conditions. For performance comparison, the fast iterative shrinkage-thresholding algorithm (FISTA) [28], orthogonal matching pursuit (OMP) based channel estimation algorithm [29], ADMM based matrix completion (ADMM-MC) [30], and recently proposed Two-Stage estimation exploiting both Sparsity and low Rankness (TSSR) [18] are selected to be carried out under the same simulation conditions. Among them, OMP is a greedy algorithm, FISTA is an 1 -regularization algorithm, where OMP requires a priori known channel sparsity and parameter for channel sparsity was set to L, and ADMM-MC algorithm is based on matrix rank priors known and parameter for channel rank was also set to L, using ADMM to recover matrix. The TSSR algorithm uses the low-rank characteristics and sparse characteristics of the millimeter wave channel at two continuous stages to estimate the millimeter wave channel.
We take the normalized mean square error as the numerical index of the estimation accuracy Fig.1 plots the NMSE of mmWave channel estimation algorithms against signal-to-noise ratios (SNR). The proposed algorithm can get a much lower NMSE and has better performance. The curve of the OMP algorithm does not vary with the change of SNR because the angle is discretized, which causes the channel sparsity to be bigger than the pre-set sparsity, that ultimately makes the OMP algorithm underfit. However, our proposed iteration method based on  ADMM can estimate millimeter-wave channel without sparsity priori. At the same time, proposed joint nuclear norm and 1−2 -regularization channel estimation can get more sparse solution compared with the 1 -regularization based FISTA algorithm. Fig.2 shows the influence of the number of transmit antennas N T at TX on the channel estimation performance. The accuracy of OMP still does not change and other methods decrease with the increase of N T . This is because as the number of antennas increase, the channel matrix dimension becomes larger, and the number of channel factor to be estimated by the methods increase. However, our algorithm has better performance than other algorithms. Fig.3 shows the influence of different numbers of training blocks on the channel estimation performance of different algorithms. The accuracy of all algorithm improved as the numbers of the training blocks is increased. Moreover, even if the number of the training blocks is very small, our proposed method still has better estimation accuracy which shows that joint nuclear norm and 1−2 -regularization based channel estimation algorithm has more reliability and stability.

V. CONCLUSION
In this article, we focused on the channel estimation of millimeter-wave massive MIMO system and proposed a joint nuclear norm and 1−2 -regularization sparse millimeter-wave channel estimation. The algorithm simultaneously utilizes the low-rank and sparse property of millimeter wave channels to provide higher-precision channel recovery. In particular, we transform the sparse millimeter wave channel estimation problem into a compressed sensing problem, use the 1−2 to enforce the channel sparsity, and combine the matrix completion theory to estimate the millimeter wave channel. The simulation results show that the performance of our algorithm significantly better than the recently proposed algorithms. (35), let (34) equal 0.