Base Station Switching and Sleep Mode Optimization With LSTM-Based User Prediction

The base station (BS) switching technique has recently attracted considerable attention for reducing power consumption in wireless networks. In this paper, we propose a novel BS switching and sleep mode optimization method to minimize the power consumption, while ensuring that the arriving user traffic is sufficiently covered. First, the user traffic in multiple time slots was predicted using the long-short term memory (LSTM) prediction model. Subsequently, we solved the Lyapunov optimization problem to obtain the optimal BS switching solution from the trade-off relationship between the reduced power consumption by BS switching and the user traffic handled in time series. Finally, we selected the sleep mode for the switched result by calculating the wake-up time and the power consumption ratio of each sleep mode. Simulation results confirm that the proposed algorithm successfully reduces the total power consumption by approximately 15% while preventing the user data queue from diverging in multiple time slots.


I. INTRODUCTION
Recently, user traffic in the wireless networks has exploded because of increase in the amount of various large-scale contents along with the commercialization of the fifth generation (5G) wireless systems. With the growth of wireless networks, green communication has attracted considerable attention for environmental protection [1], [2]. It is observed that wireless networks cover 2% of the total amount of CO 2 emissions from the earth [3], thereby facilitating global warming. In particular, operating the base station (BS) consumes 70-80% of the total energy in wireless networks [4], [5]. Therefore, recent studies have attempted to control the switching strategy of the BS to maximize energy efficiency [6]. The BS can be switched into the On/Off status to reduce its power consumption [7]. However, most studies regarding BS switching have considered only two states, either ON or OFF, and have failed to take into account the sleep mode defined by its sleeping depth. In reality, the BS sleep modes can be configured by analyzing its hardware elements [8], according The associate editor coordinating the review of this manuscript and approving it for publication was Javed Iqbal .
to the respective components operating in each mode. For example, when only a short sleeping time is allowed, we can efficiently reduce the total power consumption by switching off only the components that can rapidly change its state, which can be called ''light'' sleep mode. Consequently, we can save more energy in various scenarios by optimizing the sleep mode, compared with a scheme that uses only the ON and OFF states. Moreover, the wake-up time, which corresponds to the switching delay from operation instruction to actual operation of BS, must be considered. However, existing researches have not considered this significant factor for the practical application of BS switching.
In this study, we optimized the switching strategy of the BS (i.e., controlling the On/Off states) to reduce power consumption by considering the actual data traffic of users. Regarding the user traffic, we employed a long-short term memory (LSTM) scheme to handle dynamic traffic in a time series. LSTM [9] is a type of recurrent neural network (RNN) that feeds the output of the previous step to the input layer of the current step, which is a dynamic feedback connection, and is suitable for modeling dependencies that occur in a time series [10]. Using LSTM, we first predict user traffic VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ in a short future time slot and then solve the Lyapunov optimization problem to obtain the optimal BS switching result by analyzing the trade-off relationship between the power consumption reduced by BS switching and the user traffic to be accommodated. Finally, we optimize the BS sleep mode for multiple time slots while considering the wake-up time of the sleep modes. The main contributions of this study can be summarized as follows.
• We formulated the objective BS switching and sleep mode optimization problem by developing models of power consumption, BS sleep mode, and user traffic. The power consumption and sleep models were defined by analyzing the BS hardware elements, whereas the user data traffic was considered as a queue. Subsequently, we established an equation to minimize the power consumption while maintaining queue stability.
• The proposed LSTM-based prediction model forecasts future user traffic to reduce the overhead of passing traffic information to the BS, based on past communication information of the user, including user position, signal-to-noise ratio (SNR), and the traffic situation. Subsequently, we solved the Lyapunov optimization problem to obtain the optimal BS switching result in a time series with the predicted user traffic set. In the BS switching scheme, a user relay algorithm was proposed to handle users when the BS switches to the sleep status. Moreover, we proposed a sleep mode section algorithm by analyzing the switched result set of BSs to determine the optimal result when applying the sleep depth over multiple time slots.
• The LSTM-based traffic prediction model was compared with the linear prediction and RNN-based models. Performance evaluations revealed that the root mean square error (RMSE) of the LSTM-based traffic prediction model is smaller than that of the linear prediction and RNN-based models by up to 64.9% and 23.5%, respectively. Thereafter, we analyzed the average user data queue that converges in a stable state to ensure that user traffic is sufficiently covered. Moreover, the proposed algorithms significantly reduced the total power consumption compared with the full-operating status by 15%. The remainder of this paper is organized as follows. In Section II, we review related works on BS switching systems. In Section III, the overall optimization problem is formulated. The design of the LSTM-based user prediction is described in Section IV. Section V discusses the BS switching model by Lyapunov optimization for each time slot. Subsequently, Section VI optimizes the BS sleep mode for multiple time slots. We evaluate the performance of the proposed models in Section VII. Finally, Section VIII concludes this study.

II. RELATED WORK
In this section, we review existing related studies on BS switching systems. Unlike in the past, when performance improvement was focused on increasing wireless network traffic, green communication has recently attracted attention. The studies in [11], [12] attempted to reduce the BS power consumption, which accounts for the majority of the total network energy consumed. Wang et al. [11] minimized the BS power consumption and improved the utility efficiency of green energy by optimizing the user connection and bandwidth allocation problem. Han et al. [12] distributed the traffic load to reduce the communication overhead between the user and BS. Consequently, the power consumption of the network was significantly reduced.
Moreover, the strategies for turning off the BS have emerged to further dramatically reduce its power consumption [13], [14]. Ghazzai et al. [13] analyzed the next-generation cellular network for green communication. The authors suggested the direction of BS switching research while considering the effect on the quality of service (QoS) of users. Thereafter, Han et al. [14] analyzed various studies on BS switching strategies, such as random sleeping, distance-based sleeping, and traffic-based sleeping. To achieve this, various studies aimed to optimize the BS switching problem [15], [16]. Wu et al. [15] defined a simple BS sleep mode in a single-cell network. The BS switches into sleep mode when the system is empty, and wakes up when N defined users gather in the cell. Yang et al. [16] proposed a method to minimize the number of active BSs while ensuring the QoS of users by formulating the minimum energy consumption problem as an integer programming model. However, to apply this system to a practical situation, other factors must be considered to determine the switching states. Therefore, a switching strategy to reduce the total power consumption by analyzing the user traffic was proposed in previous studies [17], [18]. Yu et al. [17] minimized the energy cost by jointly determining the set of BSs to be activated and the transmission power level with the predictable traffic flow of users. Peng et al. [18] determined the number of BSs during peak traffic times to provide high-quality service to users; some of the BSs remain active during the idle period, whereas the remaining BSs are terminated to reduce the power consumption. Moreover, studies in [19]- [21] focused on energy efficiency through BS control by analyzing information such as user density and mobility. Cai et al. [19] aimed to minimize the power consumption of heterogeneous networks according to user uniformity. If the user set is uniformly distributed, the system searches for the optimal operating strategy by determining the switching set of small-cell BSs from the location set. If the user set is non-uniformly distributed, both user density and BS location set are considered for the switching decision. Thereafter, Feng et al. [20] maximized the energy efficiency using the game theoretical approach. They proposed a switching strategy by investigating the correlation between a user bidding strategy and BS switching cost. Gao et al. [21] proposed a switching strategy that considers the time to reach a specific BS to increase energy efficiency while observing the user mobility.
However, these previous studies failed to consider the delay when the BS switches the On/Off status, which is not realistic because it cannot be switched within such a short time. Therefore, the BS wake-up time can be considered by analyzing the hardware factors, while considering the sleep depth of BS [8], [22]. However, the switching strategy in the previous studies, including those proposed by [8], [22] can incur a significant overhead when applied to wireless networks. To overcome this, recent studies attempted to adapt a machine-learning model while establishing an efficient BS switching strategy [23], [24]. They used the compensation concept for the switching strategy through the learning model. However, they did not consider the wake-up time of the switching, and ambiguity exists regarding the user traffic being sufficiently covered.
To overcome these problems, studies in [7], [25] considered the different sleep levels and wake-up times in the system that aims to reduce the power consumption. Masoudi et al. [25] formed four different levels of sleep modes of BS to adapt in various scenarios while considering the wake-up times. Thereafter, based on the traffic model, the authors proposed reinforcement learning approach to optimize the BS switching strategy while using real mobile traffic data. However, it may vulnerable to traffic changes caused by BS switching because it is based on current user traffic. Pervaiz et al. [7] considered the multilevel sleep model to adapt to dynamic networks with fluctuating traffic profiles. The authors predicted the BS vacation time, which allows the system to minimize the effect of wake-up time while saving energy. However, it is necessary to guarantee that user traffic is sufficiently covered for practical application to real systems. The previous studies [7], [25] proposed BS sleeping strategy with divided sleep modes and wake-up times, to adapt in various real systems. However, there exist limitation that the proposed system has to control the overhead occurred at synchronizing the user traffic information. Moreover, it cannot ensure that the user traffic is sufficiently covered, which makes adapting into real systems challenging.
In this study, we predicted the user traffic for short time slots by LSTM-based model, to make our system work more worthy in the real world, by solving the limitations of previous studies. The predicted user traffic data in time series can reduce the overhead occurred at passing traffic information to the BS. Thereafter, we used the traffic data in the BS switching strategy while ensuring that the user traffic is covered, by considering the remaining data traffic of user as a queue. By expressing the remaining amount of requested traffic as a queue, it is possible to reliably guarantee the processing of user requirements than previous models that simply consider energy efficiency. Moreover, we determined the sleep mode of each BS by considering the wake-up time in a stable BS switching strategy in a time series.

III. SYSTEM MODEL AND PROBLEM FORMULATION
In this study, we considered a wireless small-cell network comprising M users and N small-cell BSs. Assuming that the Here, we aim to reduce the total power consumption of the network model while ensuring that each user's traffic is covered. To this end, we introduce the power consumption model, sleep mode of the base station, and user traffic model in the following subsections.

A. POWER CONSUMPTION MODEL
Understanding small-cell hardware is important for designing power-saving modes that can take advantage of switching off certain hardware components under low traffic. To utilize the sleep mode of BSs and calculate the power consumption of the system, we developed a hardware model for a small-cell BS designed in [26]. As described in [26], three parts exist on the hardware of small-cell BS. The first part consists of a microprocessor that implements and manages a standardized radio protocol stack and associated baseband processing as well as the backhaul connection to the core network. The second part includes a field-programmable gate array (FPGA) and other integrated circuits to implement various functions such as data encryption, hardware authentication, and network time protocol. The third part consists of radio-frequency (RF) components for transmitting and receiving data. Moreover, there is also an RF power amplifier (PA) in the third part that transfers high-power signals to the transmitting antenna.
Considering the hardware model of the small-cell BS, the largest components of the total power consumption are associated with the RF front end (45%) and the temperaturecompensated crystal oscillator (TCXO) heater (7%). This indicates that switching off these components can reduce the total power consumption by over 50%. Meanwhile, switching on the RF front end requires a few hundred milliseconds. Moreover, the TCXO also requires time to reheat. However, considering the analysis in [27], no disruption of the smallcell operation occurs, except for an induced clock drift.

B. BASE STATION SLEEP MODE
Based on the designed power consumption model, we applied the sleep mode of the BS defined in [27] to reduce the total power consumption of the system. As described in [27], the sleep modes of small-cell BS were defined, and ordered by ''depth.'' The defined sleep modes of the BS are as follows: • On: The small-cell BS is in full operation, and it consumes the maximum power.
• Stand-by: The small-cell BS is in ''light'' sleep and can rapidly wake up. The RF and TCXO heaters are turned off.
• Deep-sleep: The small-cell BS is in ''deep'' sleep, and it needs a relatively longer time to wake up. Only the power supply, backend connection, and generic CPU core are turned on. • Off: The small-cell BS is turned off, but consumes a small amount of power for activation. However, this power consumption is negligible, and is considered to be approximated to zero.
For the defined sleep modes, the wake-up times are summarized in Table 1, referring to [27]. The power consumption in Table 1 is expressed as a percentage, with respect to the amount of power consumed in the ''On'' mode, which is 100%. From now on, we will collectively call ''Stand-by'' mode, ''Deep-sleep'' mode, and ''Off'' mode as ''sleep mode'' for convenience in switching strategy. It is evident that the BS power consumption varies depending on the user traffic in the cell. However, as it is well known, the BS circuit power is responsible for most of the power consumption, and the transmission power allocation is relatively small. In addition, considering that the transmission power by the traffic is a factor that increases the complexity, but is not as impact as the switching mode factor of the BS. Therefore, we did not include the instantaneous transmission power change according to the user traffic in our power consumption model.

C. USER TRAFFIC MODEL
In this subsection, we introduce the user traffic model for formulating the BS switching problem. When switching the BS into sleep mode, the most important aspect is to confirm whether user traffic is sufficiently covered. To handle this problem, we first denote S t m as the required data amount of user m at time slot t. Thereafter, the power consumption of BS n at time slot t is denoted as P t n . Here, we define the BS switching factor as If BS n is ''Stand-by'', ''Deep-sleep'', or ''Off'' mode at time slot t, it indicates that BS n is switched to sleep mode, therefore α t n is 0. In this study, we assumed the system where the BS cells are not dense, which is disadvantage for users; thus, the system can be extended to various real systems such as heterogeneous networks (HetNet) with dense cells, and drone networks with sparse cells. To handle the traffic of users when their connected BS is turned off, the handover technique to another BS or relay technique between users can be used. In general, the BS communication has better performance than relay between users; therefore we attempted to connect to other BS first. Thereafter, we considered the deviceto-device (D2D) network, which is typically used for relaying systems [28]. We assumed the relaying system based on the model in [29] for the remaining users, as depicted in Fig. 1. In other words, if α t n = 0, user m ∈ t n employs the relay network to cover its traffic. Thus, we can define the relay factor as If β t i,j = 1, this indicates that user j requests its traffic be given to user i at time slot t because the BS to which user j belongs to is turned off. Moreover, β t i,i = 1 if user i directly receives data from the BS without using a relay.
However, although this is a system-wide gain, it would be disadvantageous for the users in terms of power consumption. According to [30], the power consumption of user increases because of the transmission process. However, it is relatively negligible compared to the power consumption of the BS; therefore we aim to minimize the power consumption of the BS, ignoring the additional energy consumed by the user, for efficiency of the entire network [31].
Thereafter, δ t n denotes the sleep depth of BS n at time slot t, which is expressed as Based on the defined factors, we express the arrival data traffic a t m,n of each user m included in BS n at time slot t as This indicates that the amount of data is delegated to the relaying entity. To make our system robust, we assume that there exists no exception of one user receiving a relay network while assisting other users as a relay. Then, the signal to interference and noise ratio (SINR) of user m included in BS n can be expressed as where p t m,n , g t m,n , and σ 2 denote the transmission power of BS n to user m at time slot t, channel gain, and the power of additive white Gaussian noise (AWGN), respectively. According to Shannon's capacity theorem, the data rate of user m in BS n, denoted as C t m,n , can be expressed as C t m,n = B n log 2 1 + γ t m,n .
Now, the departure data traffic of user m in BS n can be expressed as where B n is the bandwidth of each sub-channel of BS n.
In other words, the system allocates bandwidth depending on the number of relaying users to focus on processing of the relaying entity. In this study, the small-cell BS maintains the data queue of each user to store and forward data from the BS to the user. The queue is assumed to be simultaneously known by all BSs. We denote the queue size of user m on BS n at time slot t as Q t m,n . From the defined arrival and departure data traffic, the data queue evolves as where

D. PROBLEM FORMULATION
Based on the designed user traffic model, we formulate the optimizing problem of this system in this subsection. Our aim is to minimize the total power consumption of BSs while the user data requirement is sufficiently covered. To this end, we can express the constraint indicating that the given queue should satisfy the mean rate stability to guarantee stability as Finally, the overall optimization problem can be expressed as δ t n = {0, 1, 2, 3}, ∀m ∈ M, ∀n ∈ N, (11f) where ρ max is the limitation of the maximum QoS requirement of each user. Because we assume the relay network to supplement user traffic when the BS is switched off, and it is not directly considered in the equation, we set (11c), which holds the maximum departure requirement that can ensure the stability of the relay network. The minimum limitation of the QoS requirement of each user can be satisfied through the stability of the queue; therefore, it can be ignored.
The formulated problem (P1) is a non-convex problem by the binary terms α t n and β t m,n and the discrete variable δ t n . Moreover, equations (11a) and (11b) are the time domain objective function and constraints, which transform the problem into an NP-hard problem. To overcome this issue, we propose an algorithm that solves the Lyapunov optimization problem for the BS switching problem in Section V. We aim to minimize the total power consumption while guaranteeing VOLUME 8, 2020 the queue stability constraint by Lyapunov optimization. Subsequently, we employ machine learning techniques to achieve the original goal of reducing the total power consumption by utilizing sleep mode optimization, which is determined from sleep duration by switching results. Thus, we first predict the user traffic in Section IV.

IV. USER TRAFFIC PREDICTION
For more accurate BS On/Off and relay strategies, we use the LSTM model to predict future user traffic requirements. The LSTM model is a variation of the RNN model, which can capture the pattern from the time-series data. It attempts to resolve the vanishing/exploding gradient problem in the classic RNN model by using cell states, which include four types of gates. Fig. 2 represents the process of LSTM cell, and Eq. (12) describes the equations of the forward pass for the LSTM cell.
where x t is the input vector, f t is the forget gate vector, i t is the input/update gate vector, o t is the output gate vector, h t is the hidden state vector,c t is the cell input activation vector, and c t is the cell state vector.  The 4G trace dataset with channel and context metrics is used for the LSTM model training [32], and is collected from two major Irish mobile operators. The dataset contains 135 traces, with an average duration of 15 min per trace. Fig. 3 demonstrates the process of LSTM-based model for traffic prediction. At the training step, the input dataset is pre-processed with feature selection and normalization. The main features included in pre-processed dataset are as follows.
• Timestamp: timestamp of sample. Each sample has an interval of 1 second. • Position: each position is provided as (X, Y), which is calculated from the longitude and latitude of GPS coordination.
• SNR: SNR value at the mobile user. • Downlink traffic: downlink traffic measured at the user in Kbits.
• Uplink traffic: uplink traffic measured at the user in Kbits.
• Distance: distance between the serving cell and the user. After pre-processing, the LSTM-based model is trained with pre-processed training data. The validation loss is checked for every epoch, and the training step is completed when the validation loss increases for two epochs in a row because we consider increase in the validation loss as a symptom of overfitting. We chose the RMSE as the loss function to measure the accuracy of the prediction algorithm. Eq. (13) represents the RMSE for each sample prediction.
After the training, the testing procedure begins. For each time step, the input data x t is pre-processed similar to the training step. Future user traffic predictions for the next time steps are performed using the LSTM-based model; they are used in the process of BS switching and sleep mode optimization algorithms, as indicated in Section V and Section VI, respectively.

V. BASE STATION SWITCHING STRATEGY A. LYAPUNOV OPTIMIZATION PROBLEM FORMULATION
In Section III-D, a mathematical model for the BS switching optimization problem is designed. As shown in our objective function (P1), the proposed problem is a time-domain function, which becomes an NP-hard problem. To solve this problem, the Lyapunov drift optimization technique [33] is suitable because we can observe the trade-off between power consumption and data queue stability. In this section, the traffic is considered to be constant, according to predictions in Section IV. Then, let t denote the vector of the remaining data queues at time slot t. Thereafter, the quadratic Lyapunov function can be defined as where ( t ) T denotes the transpose of t . Then, let t be a conditional quadratic Lyapunov function, which can be formulated as E L t+1 − L t | t , i.e., the drift in time slot t [33]. The dynamic policy is designed to solve the proposed optimization formulation by observing the current data queue sizes Q t m,n and determining the switching selection of the BS to minimize a bound on [34] where V is a positive constant value parameter used to control the drift policy, which affects the reward-data queue trade-off. When we switch BSs, a reward is received. Based on the Lyapunov approach [33] with a reward for selection, we can transform the optimization problem as Finally, we can define the objective function as P2 : min Now, we solve the problem (P2) to minimize the power consumption of BSs.

B. ALGORITHM DESIGN
To solve the formulated problem (P2), we first design an algorithm for the relaying process. When we switch BS n to sleep mode at time slot t (i.e., α t n = 0), the traffic of all users in t n must be relayed to other users that are available to use their connected BS. Therefore, before designing the switching algorithm for BSs in the system, we developed a relay algorithm for users disconnected from their BS.
As depicted in Algorithm 1, BS n checks whether it is switched to the sleep mode at time slot t. For all users associated with BS n, it first determines whether user m is transferable to BS n where n = n. From [30], it is proved that communication with BS is more efficient than a relay network in terms of both energy efficiency and performance. Therefore, if the user can be transferred when considering for m ∈ t n do 3: if m can be forwarded to BS n then 4: Q t m,n ← Q t m,n . β t m,i ← 0, ∀i ∈ M. 10: Calculate Score t m,i , ∀i ∈ M, i / ∈ t n by Eq. (18). 11: Select user p in BS q that has the best score from B t m where C t p,q does not exceed ρ max after relayed. 12: 13: 14: B t m ← ∅.

15:
end if 16: end for 17: end if the coverage, all information including queue is delivered to BS n ; otherwise, user m is assisted by another relay user which is associated with BS l (α t l = 1) using D2D link. If there are multiple relay users, user m must select the most preferable relay user that has the best D2D link quality and least remaining queue size. Therefore, the preference score Score t m,i of relay user i for user m is defined by (18) where B t m denotes the set of users that user m needs to assist as a relay user at time slot t (i.e., p ∈ B t m when β t m,p = 1), γ t p denotes the SINR of shorter link of either i or m from p, j is the BS index which is associated with user i, Q max denotes the maximum size of the queue, and K is a set of users that can assist user m as a relay user. The score increases in proportion to the remaining queue size of user i and the preference of the relay network to that user. Moreover, the traffic of the user to be relayed is multiplied to obtain a a fair result. Then, in the algorithm, user m is included in B t m , and the all-relay factor from m is set to 0. Thereafter, the system selects user p that has the best score from B t m among all users not included in t n . Finally, the users in B t m are moved to B t p , which is then relayed from user p.
Because the factors α t n , ∀n ∈ N and β t i,j , ∀i, j ∈ M are closely related to each other, we have to select the initiating factor from a system perspective. To accomplish this, we sort all BSs in ascending order according to the total arrival traffic, as depicted in Algorithm 2.  N t ← Sort BSs by Arr t n in ascending order. 3: for n ∈ N t do 4: Try to switch BS n into sleep mode (i.e., α t n = 0).

5:
Select relay β t by Algorithm 1. 6: if Obj t decreases then 7: Accept switching action of BS n. 8:

9:
N t ← Re-sort BSs by Arr t n in ascending order. 10: end if 11: end for 12: if Q max − Q th < Q t m,n then V = V + V . 13: 14:

end if 15: end while
In this case, an additional amount of data from relaying can be considered by calculating the arrival traffic, as defined in (4). Therefore, using this technique, we can attempt to sequentially switch the less important BSs into sleep mode. Then, we have to select the overall relay of users in t n using Algorithm 1. To observe the queue state changed by this action, we use the objective value of Eq. (P2) as In the algorithm, we observe the change in Obj t caused by switching BS n. If the switching action induces a decrease in Obj t , the system accepts this decisive action. Therefore, the system removes BS n from the candidate BS list N t and re-sorts the set because the arrival traffic of the BSs is changed by the relay of users in BS n. However, this switching strategy cannot guarantee that the data queue will not exceed the maximum queue size Q max because the constraint (10) only observes whether the queue diverges. To overcome this, the weight factor V is varied dynamically to avoid potential queue overflow [35]. If the data queue is almost full, the weight factor V is increased to weigh the data process. In contrast, the system reduces V to achieve higher power saving of the BSs if the queue becomes stable. This procedure is depicted in line 12 to line 13 in Algorithm 2, where Q th denotes the threshold for preventing queue overflow. The varying amount V and the initial value of V are experimentally obtained to determine the appropriate values depending on the channel and queue models of this system.

VI. ADAPTIVE SLEEP MODE SELECTION STRATEGY
In this section, we select the sleep depth of the BSs in the switching result. In Section V, we optimized the switching factor α t n of the BSs and relaying factor β t i,j for all users in the time series. Consequently, we obtained a map of the overall switching result, as depicted in Fig. 4. Now, the sleep depth of BSs should be determined, which is simply decided based on whether the BS sleeps, that is, α t n = {0, 1}. Considering the defined sleep modes of the BSs in the previous section III-B, we characterize the status based on sleep duration and efficiency. Let t on , t standby , t deepsleep , and t off denote the wake-up times for each sleep mode. Hence, we can define the initial sleep mode of each BS by comparing its sleep duration and wake-up time. However, there could be a case where the initial sleep mode selection result of BS n is that from t = i to j − 1 is 'Stand-by', t = j is 'On', and j + 1 to k is 'Standby', where i < j < k. In other words, BS n can sleep deeper and longer, from time slot i to k, by switching into sleep mode at time slot j. Therefore, although the switched result is optimal in the time series, it can also be changed as indicated in time slot j in the aforementioned situation. When the mode changes, it should not be optimal in switching result, but it must be changed to become optimal result when considering the sleep mode. To this end, we first find the BS-time candidate set that can affect the sleep depth of nearby time slots. At the first use of Algorithm 3, the sleep depth of BSs is  N c,n ← Affected BS-time set from re-switching of candidate (n, t). 8: Optimize the switching problem by Algorithm 2. 9: if t≤τ ≤T Obj τ c decreases then 10: Accept switching action of BS n at time slot t.

12:
N c ← Explore candidate BS-time set. 13: end if 14: until N c is not empty set.
initialized by comparing the sleep duration and wake-up time of each mode. Then, the system explores the BS-time slot set (n, t) that exists between the long sleep conditions of BS n. To this end, we can simply use a breadth-first search (BFS) algorithm for switching maps. This is processed for all BSs, and then inserted into the candidate set N c . Subsequently, the system attempts to switch into the sleep state of BS n at time slot t. However, because the original set was the optimal switching result, the total Obj value for overall time slots along with the application of the sleeping depth should be calculated and compared. Thus, we calculate the Obj t c as where P t j is calculated by applying power consumption ratio on sleep mode δ t j . Because the determined switching set before time slot t is not affected, the system calculates Obj c after time slot t. Then, the system accepts the re-switching action if Obj c for all time slots decreases. Thereafter, the system may change the sleep mode of BS-time set N c,n , which is affected by the re-switching result of candidate (n, t). The candidate set N c is explored after re-switching, which is repeated until there exists no element in N c .

VII. SIMULATION RESULTS
In this section, we evaluate the performance of the proposed scheme, for which we developed a system model using MATLAB. In the simulations, we set the field size to 2 km × 2 km, whereas the number of users and BSs are set to 500 and 25, respectively. The maximum limitation of queue size Q max is 500, while Q th is set to 50. Furthermore, the power consumption of each BS in one time slot is assumed to be 1500 watt during the ''On'' mode. Then, the power consumption of the BS in other sleep modes is calculated according to Table 1. The maximum QoS of each relay is limited to 5 Mbps, and we assume that the bandwidth is 20 MHz for each cell. The simulation parameters are listed in Table 2. We first conduct a performance evaluation for user traffic prediction. To evaluate the performance of our LSTM-based prediction model, the following two comparison models are presented: a linear prediction model and an RNN-based model. The linear prediction model predicts future traffic with previous and current traffic values. For example, if the previous and current user traffic is 15 Mbits and 16 Mbits, predictions after one and two time slots will be 17 Mbits and 18 Mbits, respectively. The RNN-based prediction model is the same as the LSTM-based prediction model, except that the RNN cell is used instead of the LSTM cell. We used 80%, 10%, and 10% of the dataset in [32] as our training, validation, and testing data, respectively. Fig. 5 demonstrates the validation loss of RMSE during training. The linear prediction model has constant validation loss because it does not require a training step. The LSTM-based and RNN-based models converged after 15 training epochs with validation losses of approximately 2,711 Kbits and 2,897 Kbits, respectively. The validation loss of the linear prediction model is significantly higher than those of the other two models because linear prediction cannot leverage past dependencies. Fig. 6 illustrates the prediction loss of RMSE versus future time slots to predict. To leverage past dependencies, future user traffic predictions are conducted after 100 time slots. The RMSE of the three prediction models rises as the number   of future time slots to predict increases. Among the three models, the LSTM-based model demonstrated the smallest RMSE. Moreover, the RMSE differences between LSTM and other models increase as time slots to predict increases. The RMSE of the LSTM-based traffic prediction model is smaller than those of the linear prediction and RNN-based models by up to 64.9% and 23.5%, respectively. The results of our proposed model show improved performance because the LSTM can resolve the vanishing/exploding gradient problem and capture the long-term and short-term dependencies by using cell states. Consequently, it is proved that our proposed LSTM-based model predicts user traffic for future time slots more accurately compared with other models. Therefore, we can successfully expand to optimizing process of the BS switching strategy by reducing the overhead that occurs when passing the traffic information to the BS.
From the predicted user data set, we analyzed how V and V affect the Obj t for the proposed BS switching algorithm in Fig. 7. Fig. 7(a) and Fig. 7(b) present that the Obj t converges to the optimal value from Algorithm 2 with an initial value of V = 0.5 × 10 −2 and V = 0.1 × 10 −1 , respectively. It is shown that a larger value of V raises the width of the oscillation and increases the convergence speed. In contrast, a small V gradually converges with small oscillation, but it takes a long time to stabilize. This can be seen as another trade-off between the speed and stability of reaching convergence as the varying amount of the weight factor V in our Algorithm 2 is different. In other words, a large amount of V may create a slight delay to users in the beginning instead of reaching the stable point quickly. For both the V values, as depicted in Fig. 7, the Obj t value that indicates the trade-off relationship between user traffic and power consumption converges; therefore our proposed model is proved to be a stable system, which means that the user traffic can be processed within a given time while considering the actual user data traffic.  Fig. 8(a), the proposed algorithm can reduce the data queue size, considering the appropriate initial value V and varying amount V obtained by the experiment. As previously indicated, when the V value is large, it vibrates after rapidly reaching a stable state; however, the vibration magnitude is larger than the queue status with a small V . In contrast, if inappropriate V and V values are selected, the entire Obj t cannot be converged. Consequently, as shown in Fig. 8(b), the data queue diverges in an unstable state.
After confirming the stability of the proposed algorithm, as shown in Fig. 9, we show that our model reduces the total power consumption of BSs. Varying amounts are calculated using V 1 = 0.01 V and V 2 = 0.1 V . We used the accumulated power consumption, which indicates that because the time to convergence varies according to the scale of V and V , it is slightly different at the beginning and converges later. The proposed model consumes significantly less power than that consumed at the fully operating status for all simulated values. However, comparing the differences between various inputs in the graph is challenging; hence, we analyzed the average value of 500 time slots that were executed for 100 runs, as summarized in Table 3. In Eq. (20), V controls the weight between the power consumption and queue stability. Therefore, because a larger V value increases   the weight of the queue, the system aims to achieve a lower queue size compared to that obtained by a smaller V value. Consequently, as presented in Table 3, the average queue size with V = 0.1 × 10 −1 is lower than the case with V = 0.5 × 10 −2 . However, as V increases, the system has a relatively low weight for power consumption; therefore, the power consumption slightly increases compared to the case with a smaller V . Thus, compared to the power consumption when operating all BSs, the reduction ratios of cases V = 0.5×10 −2 with V 1 , V = 0.5×10 −2 with V 2 , V = 0.1× 10 −1 with V 1 , and V = 0.1 × 10 −1 with V 2 are 14.98%, 15.05%, 15.17%, and 14.65%, respectively. In other words, our proposed algorithm reduced the total power consumption by optimizing the BS switching and sleep mode efficiently while the user data traffic is sufficiently covered. Finally, we analyze the effect of Algorithm 3, as shown in Fig. 10. The wake-up time is applied to all results with the values in Table 1. When the sleep mode is not applied, most BSs cannot be switched to the sleep mode because only the On/Off states exist. When only the switching algorithm is applied (i.e., Algorithm 2 only), the power consumption is reduced, but is relatively high; this is owing to the fact that sleep depth optimization is not performed, and the system tries to optimize the switching results in each time slot, but does not consider the switching results in time series. In other words, it is not possible to optimize the overall system with only the optimal switching result. Therefore, the system can save energy as much as possible within a given sleep duration by adapting our sleep mode selection algorithm.

VIII. CONCLUSION AND FUTURE DIRECTIONS
In this study, we proposed BS switching and sleep mode optimization algorithms with an LSTM-based user prediction model. The LSTM-based traffic prediction model forecasts the future traffic of users using past communication information, including the position and traffic of users. Therefore, the system successfully reduced the overhead occurred in synchronization of traffic information between users and BSs, by forecasting future user traffic from LSTM-based prediction model. From the predicted user traffic set, a BS switching algorithm is proposed by solving the Lyapunov optimization problem, while the user traffic included in the sleeping BS is sufficiently handled by a user relay algorithm. Subsequently, the sleep depth is assigned to the switched BS result set using the proposed sleep mode selection algorithm. The simulation results confirmed that our proposed algorithms significantly reduced the total power consumption while ensuring that the user data queue converged to a stable state. In other words, our proposed BS switching and sleep mode optimization system reduced the total power consumption while ensuring that user traffic is sufficiently covered. Consequently, our model solved the limitations of existing models in terms of user traffic handling and overhead of data synchronization; therefore, the proposed BS sleeping strategy model can be expanded to various models in actual systems.
In this study, a D2D relay network with a simple handover was considered to handle user traffic. For our future research, we will extend our method to HetNet with macro cells and small cells in dense networks to build a more realistic model, while further strengthening the relaying technique by considering the handover technique. To this end, more complex queue model may needed to adapt in each layer network model. In addition, we can expand this model to save energy when using drone cells in special disaster situations for application to situations with sparser cells than those considered in this study. Moreover, the additional power consumption of each user device caused by relaying network is ignored in our proposed model, because it is relatively small when compared with the reduced energy in overall BS. Therefore, we can expand the proposed model to the system that give rewards to the users from saved power in BSs.