Energy-Efficient CoMP Joint Transmission in Hybrid-Powered mmWave Networks

In the downlink of millimeter wave (mmWave) cellular networks, coordinated multi-point transmission (CoMP) with the joint transmission (JT) strategy can achieve a higher data rate at cell edge by multiple desired signals. However, since cooperative transmissions among multiple cells lead to increased energy consumption for these cells, an energy efficient CoMP-JT strategy is required. In this paper, we propose an energy efficient CoMP-JT (EE-CoMP) scheme with hybrid power, where a small cell base station (SC) can use both renewable power and grid power. In EE-CoMP, when the multiple neighboring SCs conduct a cooperative transmission, these SCs determine whether or not to conduct CoMP-JT in a distributed manner. To minimize the grid energy consumption of SCs while maintaining the outage probability, we consider a stochastic game model with constrains for an EE-CoMP based system. In this way, the policy for conducting CoMP-JT can be obtained by applying a best response algorithm. Evaluation results show that EE-CoMP can reduce the grid energy consumption by 80% compared with a probability-based scheme while providing a sufficiently target outage probability (e.g., 0.1).


I. INTRODUCTION
In the fifth generation (5G) of cellular networks, millimeter wave (mmWave) is considered a promising technology that can meet the user demand for significantly improved performance with higher data rates [1]. Then, those demands can be achieved by a spectrum expansion for largely unused spectrum [2]. Moreover, for high traffic volumes of users in mmWave-enabled Internet of Things (IoT) networks, more small cells (SCs) are deployed to improve the capacities [3]. However, in mmWave cellular networks with dense SCs, the users around an edge of a SC still suffer from intercell interferences (ICIs) [4]. Further, the data rates can be significantly decreased by the blockage effect [5]. To overcome these problems for mmWave SC, as a cooperation diversity technique, a coordinated multipoint transmission and The associate editor coordinating the review of this manuscript and approving it for publication was Masood Ur-Rehman . reception (CoMP) has been considered [6], [7]. In particular, CoMP with a joint transmission (CoMP-JT) scheme can achieve an increased signal-to-interference and noise ratio (SINR), where the user can coherently receive the desired signal from not only its serving cell but also an adjacent cell [8].
Although CoMP-JT in dense mmWave networks can provide higher data rates to the users, it has some drawbacks for a system, such as the additional energy consumption caused by a cooperative transmission of SC [9]. Therefore, to reduce the energy consumption, in this paper, we propose an energyefficient CoMP-JT scheme (EE-CoMP) in which each SC can use hybrid power, such as both a grid and a renewable energy source [10].

II. RELATED WORK AND MAIN CONTRIBUTIONS
A number of studies have attempted to improve the energy efficiency of SC and the data rate of users in mmWave cellular VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ networks [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22]. For example, [11], [12], [13], [14], [15], and [16] introduced the dual-powered system for the energy efficiency in cellular network. In [11], the authors showed that dual-powered system can provide a seamless operation of the base stations (BSs) without energy outages. Balakrishnan et al. [12] showed the impact of dual-powered BS for traffic-energy imbalance. Al Haj Hassan et al. [13] considered stochastic environments such as harvested energy and channel conditions for dual-powered networks. The results from [12] and [13] demonstrated that the dual powered system can increase the revenue gain while guaranteeing user requirements such as delay constrained and tolerance. In [14], the authors proposed an energy sharing based cooperative framework for dual-powered BS wherein a BS that is not experiencing an energy outage can conduct the energy cooperation. Li et al. [15] designed a energy efficient mmWave cellular networks where SCs harvest electromagnetic energy and optimized the transmission power of SCs by formulating a fractional programming problem and exploiting several mathematical methods (i.e., Dinkerbach and Lagrange dual decoupling). Liu et al. [16] proposed a cost-aware online resource management algorithm for energy cost minimization for cellular networks with renewable and grid energy power. The results showed that the proposed algorithm could determine the optimal solution between the energy cost and the queue backlog.
To increase the data rate of users in mmWave cellular networks, [17], [18], [19], [20], [21], [22] considered cooperative transmission technology. In [17], the authors showed the impact of a dual link by cooperative transmission for blocking objects such as stochastic geometry and random shape. Gerasimenko et al. [18] established the user performance model for the connectivity strategy in dense mmWave SC, which is determined according to SC density and height as well as blocker density. The results showed that the multi-connectivity mechanism in mmWave SCs can improve the capacity of the user. In [19], the authors proposed a dynamic control algorithm to improve the power-efficient, highly reliable, and low-latency, where the CoMP serving set from SCs is selected dynamically by exploiting the queue and channel states. Zhao et al. [20] introduced the three CoMP strategies to improve the coverage probability and the mean local delay in the mmWave cellular networks, where the three strategies are fixed-number BS cooperation (FRC), fixed-region BS cooperation (FNC), and interference-aware BS cooperation (IAC). The results showed that the performance of the IAC outperforms other schemes for the high density of BSs. Arfaoui et al. [21] introduced two multiple access schemes using zero-forcing pre-coding technique to increase the data rate of users. In [22], the authors proposed a CoMP enabled heterogeneous network with a hybrid-powered to maximize the system energy efficiency, where hybrid power is allocated by considering the energy consumption.
However, no previous work has optimized the energy efficiency and outage probability for mmWave networks using CoMP-JT in a distributed manner. 1 In our proposed scheme, when the users require increased data rates, the additional assistance of multiple SCs (using CoMP-JT) is conducted, where each SC can use hybrid power such as grid and renewable power. However, the excessively blackundant cooperative transmissions of the SCs can lead to diminished energy efficiency. Therefore, in our scheme, the cooperative SCs determine whether or not to conduct CoMP-JT in a distributed manner, where the grid energy consumption and the outage probability of SCs are considered. To optimize the trade-off between the grid energy consumption and the outage probability, we formulate a constrained stochastic game model. The solution of the game model is obtained using a best response algorithm. The results show that EE-CoMP can decrease the grid energy consumption compared to that of a probability-based scheme while providing a sufficient target outage probability.
The contributions of our paper can be summarized as follows: 1) The energy efficient mmWave network system using CoMP-JT is proposed in which cooperative SCs selectively conduct CoMP-JT; 2) Unlike the conventional centralized approaches, for an EE-CoMP-based system, we obtain the optimal policy regarding the behavior of cooperative SCs in a distributed manner, where a stochastic game model with constrains is formulated; and 3) We evaluate EE-CoMP under various environments, which are used to provide guidelines for the design of EE-CoMP-based mmWave systems.
The rest of this paper is organized as follows. Section III describes the EE-CoMP-based mmWave system. In Section IV, the constrained stochastic game model is formulated. Section V presents the numerical examples, and Section VI provides concluding remarks. Fig. 1 shows the two scenarios of an EE-CoMP-based mmWave network system wherein, since each SC has energy harvesting capability, renewable power and grid power can both be used. Moreover, we consider the downlink of mmWave cellular networks where SCs use CoMP-JT to achieve a higher data rate. That is, the user can receive the desired signals from multiple SCs. In scenario (a), SC 1 and SC 2 conduct the CoMP-JT while, since SC 3 does not have renewable energy, it does not conduct the CoMP-JT. Further, although SC 4 does not have renewable energy, it conducts CoMP-JT to provide higher data rate for the user. Thus, the grid energy of SC 4 is used. For a scenario (b), the user can receive the desired signals from SC 1 and SC 2 with renewable energy. On the other side, although SC 4 has renewable energy, it does not conduct the CoMP-JT to the user, where the desired signal power level for the user only satisfies two 1 To optimize the energy efficiency and outage probability for mmWave networks, we can introduce a centralized approach where a central server (e.g., central cloud) can collect all state information and derive the optimal stationary policy of all SCs. However, this approach may lead to high signaling overhead and a single point of bottleneck/failure. Therefore, we introduce a distributed approach. signals. Thus, in this case, the grid and renewable energies can both be saved.

III. SYSTEM MODEL
As shown in these scenarios, several SCs can conduct CoMP-JT to the user (e.g., SC 1, SC 2, and SC 4 in scenario (a)). However, in scenario (a), if it is assumed that the user has a sufficient SINR even without CoMP-JT of SC 4, CoMP-JT of SC 4 can be considered as unnecessary transmission. Moreover, in this scenario, since SC 4 conducts CoMP-JT by using the grid energy, lower energy efficiency is inevitably achieved. To mitigate this problem, in our proposed system, each SC decides whether or not to conduct CoMP-JT by considering the actions of other SCs. In so doing, as in scenario (b), if the user can have a sufficient SINR from the CoMP-JT of SC 1 and SC 2, SC 4 does not conduct the CoMP-JT to the user although SC 4 has renewable energy. Then, this saved renewable energy can be used for subsequent transmission. Note that, for the optimal decision on whether or not to conduct CoMP-JT by considering the actions of other SCs, we introduce the stochastic game model, which will be elaborated in Section IV.
In our system model, we consider the outdoor dense mmWave networks. For a propagation model, the signal power received at a user from SC i, i.e., P i , is expressed as where P t is the transmit power of SC, and r −α i denotes a path-loss at a distance r i between SC i and a user, where α denotes the path-loss exponent. Further, h i indicates the small-scale fading, which is a normalized Gamma random variable with parameter m, i.e., h i ∼ (m, 1/m). Note that mmWave links are generally well-modeled by Nakagami fading [23].
For a conventional cellular network system, according to Eq. (1), the SINR of the average power received at a user is as follows: where P 0 is the received signal power from a target SC, I and σ 2 respectively denote the sum of interference power and additive noise.

IV. STOCHASTIC GAME MODEL AND OPTIMIZATION FORMULATION
In a stochastic game model with constraints, SCs and players are used interchangeably. Therefore, in our formulation, SC i and player i have the same meaning.

A. LOCAL STATE SPACE
Each SC i has a finite local state space S i . The local state space, S i , can be defined as where E i and N i denote the renewable energy levels of SC i and the state space for the number of users in SC i, respectively. E i can be defined as where E max denotes the renewable energy capacity of SC i. N i can be represented by where N max denotes the maximum number of users in SC coverage. Note that N max depends on the capacity of SC i. In addition, S = i S i denotes a global state space, where is the Cartesian product. Then, S −i = j = S j is the state space of all SCs except SC i. VOLUME 10, 2022

B. ACTION SPACE
Since SC i can decide whether to conduct CoMP-JT or not, a finite local action set of SC i, A i , can be represented by In Eq. (7), for P[E i |E i , N i , A i ], we assume that the harvested energy k during the decision epoch at SC i follows the Poisson distribution with mean λ E , P E (λ E , k) for the tactical modeling as in [29]. 2 That is, we assume that when SC i does not conduct CoMP-JT (i.e., A i = 0), the renewable energy level of SC i in the next state, E i , increases by k with the probability P E (λ E , k), where SC i cannot store renewable energy exceeding its capacity (i.e., E i ≤ E max ). Therefore, the corresponding probabilities can be defined as Eq. (8), shown at the bottom of the next page.
Meanwhile, for N i number of users, when SC i conducts CoMP-JT (i.e., A i = 1), E i decreases by N i , where if there is no sufficient renewable energy in SC i (i.e., E i < N i ), then the grid energy in SC i is used as often as the insufficient renewable energy and E i = 0. Therefore, the corresponding probabilities can be defined as Eq. (9), shown at the bottom of the next page.
Moreover, when we assume that the arrived users k during decision epoch in SC i follows the Poisson distribution with mean λ U , P λ (λ U , k), P[N i |N i ] is defined by where N i ≤ N max .

D. COST FUNCTION
To minimize the grid energy consumption of SC i, we define the cost function v(S i , A i ), where when SC i conducts CoMP-JT (i.e., A i = 1) and does not have sufficient renewable energy to service for users, n i , (i.e., E i − n i < 0), the grid energy is used. That is, this case occurs when E i < n i . Therefore, the average used grid energy of this case can be calculated as Eq. (10), shown at the bottom of the next page, where |x| denotes the absolute value of x.

E. CONSTRAINT FUNCTION
To maintain the average outage probability, we define the constraint function c(S i , A i ), where it is defined that the outage occurs when the SINR is under threshold γ th . For n neighbor SCs which can conduct CoMP-JT, the outage probability can be represented as Eq. (11), shown at the bottom of the next page, where λ −i denotes the probability that at least one SC except for SC i conducts CoMP-JT. Further, B(j, λ −i ) = P t r −α j h j δ j , where δ j denotes a delta function to return 1 if SC j conducts CoMP-JT with λ −i . Otherwise, the function returns 0.

F. OPTIMIZATION FORMULATION
The stationary multi-policy of all SC, π , is to be optimized such that the long-term average grid energy consumption and the long-term average of outage probability constraint are both met. For this purpose, the long-term average grid energy consumption, ψ G (π ), and the long-term average of outage probability, ψ o (π ), are defined as and respectively, where S t ∈ S and A t ∈ 0, 1 are the global state and action at time t, respectively. From Eqs. (13) and (14), the constrained stochastic game is formulated as follows: where γ o is the target average outage probability. The constrained Nash equilibrium is the solution of the formulated constrained stochastic game. Let π * be the best response policy, which is the constrained Nash equilibrium when the following condition is satisfied.
where π i and π −i respectively denote a stationary policy of SC i and a stationary policy of all SCs except SC i, and π * = (π i , π −i ). To obtain π * , we apply a linear programming (LP) approach. Let ϕ i,π −i (S i , A i ) be the stationary probability in local state S i and action A i of the SC i given the policies of the other SCs π −i . Then, the equivalent LP model can be formulated and its solution ϕ i,π −i (S i , A i ) can be interpreted as the optimal policy of the formulated game [24], [25]. The LP problem corresponding to Eqs. (15) and (16) can therefore be expressed as follows: In this optimization problem, Eq. (18) is the objective function used to minimize the average grid energy consumption of SC i, and the constraints are expressed as Eqs. (19), (20), (21), and (22). Specifically, Eq. (19) implies that the target outage probability should always be kept lower than the desired target outage probability γ o . The constraints in Eqs. (20), (21), and (22) respectively represent the Chapman-Kolmogorov equation and the probability properties.
The stationary best response policy of SC i can be obtained as follows: The constrained Nash equilibrium ensures that SC i cannot achieve a lower cost by adopting any other stationary policies while the other SCs do not change their stationary policies. To update the policies of SCs as shown in the algorithm, we apply best response dynamics [26], [27]. Note that an interaction among SCs is conducted through the probability λ −i . Then, λ −i can be obtained by Calculate λ −i from (23) 6: Obtain π i * by solving the LP problem 7: end for 8: until Convergence of the policies for all SCs where λ i denotes the probability that SC i conducts CoMP-JT; this can be expressed as

G. BEST RESPONSE ALGORITHM
As shown in Algorithm 1, to begin, each SC initializes its policy (line 1 in Algorithm 1), then broadcasts its policy and CoMP-JT conduction probability to neighbor SC (line 4 in Algorithm 1). After receiving the information, each neighbor SC calculates the probability that at least one SC except SC i conducts CoMP-JT (i.e., λ −i ) given the policies and CoMP-JT conduction probabilities (line 5 in Algorithm 1). Then, each SC solves the LP problem to obtain the best response policy π i * (line 6 in Algorithm 1). 3 This procedure is repeated until the stationary policies of all SCs converge.
(11) VOLUME 10, 2022  The complexity of the proposed algorithm depends on the complexity of solving LP problem. The complexity of the Vaidya's algorithm, that is the representative algorithm solving a LP problem, is O((|S i ||A i |) 3 ) [28] (i.e., polynomial time), where |S i | and |A i | denote the numbers of states and actions of SC i, respectively. Meanwhile, to reach the equilibrium, the algorithm needs only a few iterations (see Fig. 3). To sum up, the proposed algorithm can easily be implemented in real CoMP-JT systems.

V. NUMERICAL EXAMPLES
To evaluate the performance of the proposed system, we built a simulation program using MATLAB. In the downlink of mmWave cellular networks, the number of SCs that can conduct CoMP-JT is set to 4 and the interfering SCs are set to 15. To consider the outdoor dense mmWave networks, as shown in Fig. 2, the distance between user and SCs conducting CoMP-JT, r c = 200m, and the distance between user and interfering SCs, r i = 300m [17], [20]. For the channel model of mmWave networks, we consider that each SC has i.i.d. Nakagami fading channel and m = 3 [23]. Moreover, α = 2, γ th = 1 and P t = 30dBm. Table 2 lists the chosen system parameters for mmWave networks [20]. In addition, our system is compared with four schemes as follows: 1) Always, where SCs always conduct CoMP-JT; 2) P-based, where SCs conduct CoMP-JT with the probability P, where P is set to 0.8; 3) Rand, where SCs randomly conduct CoMP-JT; and 4) Non-CoMP, where SCs do not conduct CoMP-JT.   3 shows the process wherein the policies of SC converge to the constrained Nash equilibrium policy. We assume that, for the initial probability to conduct CoMP-JT, SC 1 has 0.9, SC 2 has 0.5, SC 3 has 0.1, and SC 4 has 0.8. As can be seen in Fig. 3, the best response algorithm converges within a few iterations (i.e., three iterations). This means that our scheme can be implemented into the system without high overhead. We can see that each SC chooses its action by considering the other SCs actions at the best response. After the convergence, SC 1 and SC 4 with the high initial probability to conduct CoMP-JT obtain the low probability. In so doing, their grid energy consumptions can be reduced. Meanwhile, SC 2 and SC 3 with the low initial probability to conduct CoMP-JT have high probability to maintain the target outage probability. Fig. 4 and Fig. 5 show the changes in the average grid energy consumption ψ G and the outage probability ψ o according to N i , respectively, where N c = 3. From these figures, we can see that our system operates adaptively in varied N i , where, as N i increases, the average grid energy consumption of EE-CoMP increases while keeping the target outage probability (i.e., γ o ). This is because SCs in EE-CoMP determine whether or not to conduct CoMP-JT while considering other SCs' policies. For example, for N i = 8, ψ G = 0.01 and ψ o = 0.1. This means that, even if a small number of SCs participate in conducting CoMP-JT, it can maintain the target outage probability. Since SCs in CoMP-JT recognize this fact, most of them do not participate in conducting CoMP-JT, which leads to diminished grid energy consumption (i.e., higher energy efficiency). On the other hand, for N i = 16, to meet the target outage probability, the probability to conduct CoMP-JT of SC can be increased. Thus, the grid energy consumption is increased. On the other hand, for the other compared schemes, as N i increases, ψ G of these schemes increase as well. Because they do not adjust their conducting CoMP-JT policies by considering the interference level, their grid energy consumptions remain constant regardless of N i , and their outage probabilities increase as N i increases. We can also see that even if non-cooperation transmission scheme   (i. e., Non-CoMP) offers the low grid energy consumption, this scheme suffers from insufficient average outage probability. Specifically, CoMP-JT has an improved average outage probability when the interference power level is high. Fig. 6 shows ψ G with respect to ψ o , where EE-CoMP (0.1) and EE-CoMP (0.2) respectively denote γ o = 0.1 and 0.2.  From the vertical red line of Fig. 6, we can see that EE-CoMP can prolong ψ G above ψ o . That is, as N i , ψ G are increased and ψ o is maintained. This is because the SCs in our system can energy-efficiently operate (i.e., SCs can reduce unnecessary conducting CoMP-JT) by considering the actions of neighbor SCs. We can also see that ψ G cannot be decreased without increasing the tolerance of the target outage probability in the system. For example, when γ o =0.1, ψ G increases from 0.01 to 0.24 with increasing N i . Meanwhile, for γ o = 0.2, ψ G can be maximized at 0.03. Fig. 7 and Fig. 8 show the effect of the number of SCs that participate in the conducting CoMP-JT, i. e., N c , on ψ G and ψ o , respectively, where N i = 15. From these results, we can see that EE-CoMP operates adaptively when N c is changed. That is, as N c increases, ψ G is decreased and ψ o is maintained. This is because the probability to conduct CoMP-JT of SC is decreased with increasing N c . As a result, the grid energy consumption is decreased. For example, for N c = 3, ψ G = 0.12. When N c = 7, ψ G = 0.01. We can also see that the ψ G of other schemes remain constant regardless of N c , because those policies are fixed. Then, the outage probabilities of other schemes decrease as N c increases.

VI. CONCLUSION
In this paper, we propose an energy efficient CoMP-JT (EE-CoMP) scheme for the outdoor dense mmWave networks. In our scheme, SCs in the cooperative region decide whether or not to conduct a CoMP-JT by considering the actions of other SCs in a distributed manner. To minimize the grid energy consumption of SCs while maintaining the target outage probability of the system, a stochastic game model with constraints is formulated and a multipolicy is obtained by a best response algorithm. The results show that EE-CoMP can reduce the grid energy consumption by 80% compared with a probability-based scheme while providing a sufficiently target outage probability (e.g., 0.1). In our future work, to overcome the limitation (that each SC should have some statistical information) of the proposed scheme, we will introduce a multi-agent reinforcement learning approach.