Energy-efficient ultra-dense 5G networks: Recent Advances, Taxonomy and Future Research Directions

The global surge of connected devices and multimedia services necessitates increased capacity and coverage of communication networks. One approach to address the unprecedented rise in capacity and coverage requirement is deploying several small cells to create ultra-dense networks. This, however, exacerbates problems with energy consumption and network management due to the density and unplanned nature of the deployment. This review discusses various approaches to solving energy efficiency problems in ultra-dense networks, ranging from deployment to optimisation. Based on the review, we propose a taxonomy, summarise key findings, and discuss operational and implementation details of past research contributions. In particular, we focus on popular approaches such as machine learning, game theory, stochastic and heuristic techniques in the ultra-dense network from an energy perspective due to their promise in addressing the issue in future networks. Furthermore, we identify several challenges for improving energy efficiency in an ultra-dense network. Finally, future research directions are outlined for improving energy efficiency in ultra-dense networks in 5G and beyond 5G networks.


I. INTRODUCTION
The world is changing continuously with the emergence of new technologies, a high degree of automation, growing data rates, ultra-reliable low latency and massive machine-type communication. According to CISCO annual report [1], data traffic is expected to increase 46% from 2018 to 2022, and it will eventually reach 77 exabytes per month by the end of the year 2022. Besides data consumption, the wireless network users will reach 29.3 billion per capita with 14.7 billion machine-to-machine (M2M) connections in 2023 [2]. The wireless industry is overgrowing rapidly to accommodate future communication requirements that consist of various devices with varying service requirements from low latency to high-speed constraints, different features, and pervasive connectivity [3].
5G road-map envisions target that includes 10-100x peak-rate data rate, 1000x network capacity, 10x energy efficiency, and 10-30x lower latency, paving the way towards Gigabit wireless. In order to address the diverse requirement, 5G use cases have been grouped into three categories: • eMBB: enhanced mobile broadband (eMBB) network must provide high data rate, extensive coverage and connectivity [4]. • URLLC: ultra-reliable low latency (URLLC) optimise the throughput and delay of the communication network. Some of its use cases are autonomous/unmanned vehicles, smart grids, medical surgeries and robotics. • mMTC: massive machine type communication (mMTC) supports a large number of low-cost devices that require a low data rate for communication or having small data payloads.
In order to meet the requirement of various use cases, several new technologies are added to improve the coverage and cope with user data demands. These techniques can be categorised as: • Spectral efficiency enhancement: Spectral efficiency VOLUME 4, 2016 is considered a key performance metric to determine how efficiently the spectrum is utilised (measured in bits/s/Hz). The enhanced architecture and advances in multiple-input multiple-output (MIMO) communications provide higher spectral efficiency by focusing on narrow beams. Some other technologies such as sparse code multiple access (SCMA), universal filtered multi-carrier (UFMC) and some novel channel coding schemes like low-density parity-check code (LDPC), polar codes also help to increase the spectral efficiency [5]. Further, radio resource management also allows for improved spectral efficiency through dynamic channel allocation, changing the link and power control. • Efficient utilisation of spectral resources: Another effective way to add capacity is by making dynamic usage of spectral resources. Due to limited spectrum resources, it is challenging to meet the growing data demands. Managing frequency spectrum dynamically beyond microwave and millimetre-wave (mmWave) spectra can improve spectral resource utilisation. Currently, 5G release 16 supports 52.6GHz carrier frequency; however, growing traffic demands more bandwidth in terms of tens of gigahertz (GHz) or up-to terahertz (THz) may be required to provide services of up to 100Gbits or more [6]. Moving towards the higher frequency range in the terahertz domain is the necessity of 5G and B5G networks to assure ultra-dense network traffic demands [7]. • Increasing the density: Another way to increase the network capacity is increasing the site density by deploying several small cells. Network densification reduces the latency and improves geographical coverage by adding extra capacity, benefiting growing end-user demands. In addition to the technologies mentioned above, other qualities like virtualisation, pliable network, self-organisation, and network scalability are essential for realising the 5G vision [8].
Adding more bandwidth is expensive due to acquiring the license and the path loss associated with the higher frequency. Furthermore, point-to-point link throughput has almost reached its theoretical limit due to scarcity of available bandwidth and increased usage of available spectrum [9]. Due to growing traffic, higher spectrum utilisation has reached its saturation point in terms of available frequency per cell. Due to this, there is very little room for further significant improvement in its performance. The small cell deployment related to eMBB and massive MTC 5G satisfies the critical performance indicators of spectral efficiency, energy efficiency, and latency. It allows for higher frequency reuse, increases the capacity and data rate in the network. Also, low power small cell deployment has become popular because of the growing demands and increased power consumption. Small cells help increase the throughput and significantly reduce power consumption as dense deployment decreases the distance between the user equipment and base station (BS). Even though densification is an easy way to achieve high data rates by minimising the distance between nodes, it leads to severe inter-cell interference and energy efficiency (EE) challenges. In addition, even if small cells consume less power than traditional base stations, the cumulative power consumed by a large number of small cells results in increased energy consumption due to the additional construction and circuit power requirements [10]. Since these issues are prevalent, energy efficiency has become a priority in 5G network research.
According to a study [11], 5G consumes almost 1200W to 1400W of power which is almost 300% more energy as compared to 4G. Estimated power consumption of different network generations, in terms of carbon footprint, power consumption, radio access network (RAN) electricity consumption, and operating expenses (Opex), is shown in Table 1. Base stations are the major contributors in energy consumption which is around 80%, the rest 20% is due to cooling systems, power amplifiers, and other electronic devices [12]. Despite the high cumulative energy consumption of ultra-dense network (UDN) due to extensive infrastructure, energy efficiency can be improved by appropriately switching on/off cells based on traffic conditions [3].
A. MOTIVATION An increasing number of online services and growing user data needs are resulting in 0.5% of the world's energy consumption by mobile networks alone [14]. The spatial densification of base stations (femtocells, picocells and relays) highlights more complexities in UDN. In addition to increasing OPEX costs, its unplanned deployment results in increased interference levels and energy consumption. According to a recent report by Ericsson, [15], it is predicted that the amount of user data will reach four times more in 2025 than today's network. 5G networks consume almost four times more energy than 4G [16]. From the ecological perspective, greenhouse gas emissions (GHG) from information and communications technology (ICT) put together the major share in global warning [17]. It is estimated that UDN carbon dioxide (CO 2 ) emission will contribute to 14% in year 2040, in global GHG emission [18].
In addition to energy consumption, interference management, resource allocation, and quality of service are also essential factors in UDN. Effective resource allocation helps to determine the stability and efficiency of network performance. A resource allocation strategy needs to be fair enough not to degrade any performance metric. Sub-optimal resource allocation can further degrade the energy efficiency and channel conditions. Especially in the case of UDN when spectrum access is overlaid, which causes poor bandwidth allocation to small cells.
Along with resource allocation challenges, the inter-  1: Estimation of consumed energy in ICT, data centers, carbon footprint, radio base station and core network for the time period from 2001 to 2030 [13] ference issue is also a significant problem, especially in UDN, for two significant reasons. One is because of the dense deployment of small cells, which may affect the performance of neighbouring cells. Second, due to their adjacent deployment, which causes more mutual interference than traditional networks. Also, the network topology of UDN small cells is considerably different as the small cells are generally deployed in an unplanned manner. These factors lead the interference management paradigm exceptionally challenging.
The complexity of small cell deployment in UDN leads to significant mobility management issues as well. The variation in transmission power of macro base station (MBS) & small base station (SBS) and the network interference causes quality of service (QoS) degradation. As the SBS has less coverage area than MBS, the cell selection and re-selection becomes challenging because of more available BSs. UDN is crucial for 5G and beyond networks due to its capacity and coverage improvement and low cost of implementation. Several works have been done on UDN, but several issues, significantly improving energy efficiency, are yet to be addressed.
Several studies have been conducted in the past to improve UDN performance and energy efficiency. This article aims to review promising state-of-the-art techniques applied to solving energy efficiency issues in UDN.

B. CONTRIBUTION
Various surveys have been carried out on UDN. However, our paper is different from [19]- [25] as we aim to provide a comprehensive survey on energy efficiency in the UDN considering the recent advances focusing on promising techniques such as machine learning, game theory, other optimisation & heuristic techniques. In addition, this paper also discusses the future work and research gaps of past researches. This survey is one of its kind, which presents the taxonomy on ultra-dense networks to improve energy efficiency and motivation to select the right tool to solve the energy efficiency problem. Table  2 represents a summary and research gaps of the past surveys on UDN. In particular, the main contributions of this paper is as summarised below: 1) A comprehensive survey has been presented by iden-tifying the key elements of the ultra-dense network and taxonomy along with the constraints, optimisation types, latest modelling techniques/algorithms for 5G ultra-dense networks.
2) A detailed taxonomy has been presented on the optimisation problems in UDN by categorising them into six approaches, specifically, resource allocation, cell association/selection, interference management, base station switching, cell zooming and traffic & mobility. 3) Further, a comprehensive study has been presented on the latest modelling techniques and how those tools will help achieve energy efficiency in the ultradense network without compromising QoS. The discussed techniques are machine learning, game theory, other optimisation & heuristic techniques. 4) Various involved performance measures, EE metrics are discussed that are important to evaluate the performance of UDN. The energy efficiency metrics are further categorised into the component level, node level and network level. 5) Finally, the future research directions and challenges that can further pave the research activities in ultradense networks and related domains are provided.

C. ORGANIZATION
The rest of the paper is organised as follows: Section II presents the comprehensive survey of UDN, their architecture and components. The emerging techniques and tools (machine learning, game theory and optimisation & heuristic approaches) are discussed in Section III-D that can be used in UDN to achieve energy efficiency. Section III contains a detailed taxonomy of UDN to improve energy efficiency through resource allocation, cell association/selection, interference management, BS switching, cell zooming and traffic & mobility approaches. It also covers the performance measures, energy efficiency metrics, and latest mathematical tools discussion about improving the energy constraint through mentioned approaches. Finally, section IV concludes the paper with future research directions and challenges to extend the research work in UDN. VOLUME 4, 2016 Reference Scope Focus Limitations [19] Energy efficiency strategies for 5G green network An energy-efficient architecture was presented in the paper, and energy efficiency was discussed for the overall 5G network.
The article lacks a detailed discussion of energy efficiency in the UDN.
A survey on UDN was presented; it is more focused on the basics and technologies of UDN. Backhauling and other research directions are also discussed in this paper.
The survey lacks a discussion on machine learning and other latest modelling techniques. [21] A detailed discussion on ultra-dense heterogeneous networks (HetNet) concerning big data.
This article discusses big data aware framework design guidelines to reduce the energy cost in ultra-dense HetNet. The survey is based on base stations, data and resource analysis and caching.
The focus of the article is only on the framework part of ultra-dense HetNet. [22] A detailed survey on M2M communication in UDN.
This survey focuses on the importance of M2M communication in UDN. A detailed survey has been presented on the implementation, architecture and different layers of M2M in UDN.
The paper lacks spectral and energy efficiency part, which is important in any network.
[23] Security issues in UDN. A detailed discussion on security issues in UDN. The paper lacks systematic analysis and detailed solutions for security issues. [24] Deployment & usercentric design.
A discussion on the user based deployment in UDN.
The paper lacks taxonomy and discussion on modelling techniques related solutions. [25] Opportunities in UDN with enabling technologies.  In an ultra-dense network, conventional high-power macrocells coexist with several low-power small cells. These small cells can be mixtures of picocells, femtocells, radio remote head (RRH) and relay nodes. The emergence of UDN and small cells helped lay a foundation for fast, sustainable networks, resulting in mobile broadband (MBB) traffic growth. UDN allows for flexibility and enables the following trends for future networks: 1) Small cells can facilitate site deployment by providing wireless backhaul services. 2) UDN can create a new industry ecosystem as small cells can be deployed adjacent to end-users. 3) Enables new use cases and enables a wide array of services in 5G. 4) Support for licensed and non-licensed spectrum.
Over the past years, the commercial deployment of small cells has proven itself as an optimal solution to improve user experience as UDN takes advantage of the existing spectrum by deploying more cells [26].

A. COMPONENTS OF UDN
Networks defined as ultra-dense consist of more deployed cells than users active in the network [20]. The reason behind this is to provide high system throughput, capacity and coverage. UDN generally consist of fully functional BSs providing varying coverage areas. A BS can be a macrocell providing broad coverage and serving as an anchor or a small cell that provides high throughput. Relays and RRH can be deployed as an extension to macrocells.
These BSs (femtocells and picocells) are fully equipped to provide all functionalities of macrocells while consuming low power. However, RRHs and relays perform limited functions. The apparent difference among different BS is size, coverage, power consumption and deployment cost. Table 3 shows macrocell and small cell deployment statistics in UDN. In the following, we define different types of cells and their functionalities:  Apart from macrocells and small cells, a typical BS contains other transceiver components as well. Among the multiple transceivers (TRXs) of BS, each consists of a power amplifier (PA), baseband unit (BB), a cooling system and a radio frequency (RF) transceiver. For the power supply, a DC-DC and AC-DC unit is also installed for power grid connectivity. The transmission power and coverage for both relays and picocells are the same; however, they vary due to their functionalities, extension perspective and backhauling. Relays are used for coverage area extension, but picocells are for improving capacity. The backhauling is also different because of wired and wireless connectivity in pico and relay, respectively. The connectivity among picocells to the core network is through microwave backhaul or fibre, whereas relays are connected through radio interface [27]. The main difference between picocell and relay backhauling is because of their infrastructure management. Picocells are part of the infrastructure, whereas relays are independent and do not require configuration (frequency, neighbours). Backhaul deployment for picocell is another concern because of the growing number of picocells network point of view [28]. In the case of picocell, backhauling, proper deployment planning and designing are needed in terms of channel allocation [29]. The UDN fulfil stringent coverage and capacity requirements via small cells deployment. These small cells are manifested to remain an integral part of future wireless networks.
Theoretically, the capacity of the wireless network scales with the number of small cells as reducing the cell radius to pack more small cells results in more efficient spectrum utilisation due to higher frequency reuse. In practice, the resulting UDN poses several challenges as cells adjoin. Apart from improved received signal strength (RSSI), dense deployment of small cells cause severe interference because of increased overlapping. The initial deployment phase of 5G adopted an orthogonal deployment approach to increase capacity. Orthogonal deployment was used to reduce the interference with macrocells and achieve significant gains through spatial reuse. The second deployment phase focuses on improving peak data rates using mmWave frequency bands that can provide up to 10Gb/s of data transmission [30]. In addition to interference challenges, increasing network scale with small cell deployment introduces new energy considerations not addressed in 5G [31]. In response to increasing data and traffic volumes, efficient optimisation techniques for the small cell network are essential. The energy efficiency techniques are categorised based on network components. There are physical (mobile backhaul, base stations, femtocells, and user equipment) and technical network components involved, requiring planning, coordination, and estimation of adjustment time. Table 4 shows significant components that can be optimised to improve energy efficiency. VOLUME   The widespread deployment of UDN poses many challenges, particularly when energy consumption is the main concern. Hence, • An effective coordination scheme is required to decrease the interference between cells and crossnetwork. Spectrum reuse can significantly degrade network energy efficiency. • Proper planning is a must for BS deployment as in worst scenarios, there will be huge energy consumption. As UDN traffic load differs in both temporal and spatial domains.
This section presents the taxonomy of ultra-dense network characteristics in 5G as shown in Figure 1. The rest of the section explains the detailed taxonomy that highlights popular approaches used over the past years to address energy efficiency issues in UDN, the latest used modelling techniques, metrics to measure performance, and objectives to improve energy efficiency.

A. NETWORK PLANNING & ARCHITECTURE
The existing wireless network architectures emphasise voice and mobile broadband services. However, the trend changed with the transition of demands from coverage to capacity. To meet future needs, the way that cellular networks are implemented needs to change because: • The diversified 5G network consists of multiple BS types (macrocells and small cells) and supports existing standards & multi-connectivity. It also becomes difficult to support such huge, diverse network requirements like throughput, connectivity and mobility. • There are different service requirements for different functions, such as high throughput for enhanced mobile broadband services, ultra-low latency for ultra-reliable low latency communications. • The growing demands are resulting in frequent new services provisions which require rapid deployments. Also, an improved management operation is required for deployment, maintenance and implementation.
In the initial stages of network densification, more macrocells were installed to increase coverage and capacity simultaneously. However, the development of indoor hotspots became necessary with the increasing number of indoor devices. The hotspots were confined to a specific area and dispersed throughout, but researchers developed small cells of today after that concept. The dense network consists of several small cells (i.e. microcell, picocell and femtocell) to provide a dense heterogeneous network. The critical factor in planning a wireless network is finding out the correct coverage area since the exact number of deployed BS rely on the coverage area of interest [32]. Moreover, planning a network and deploying the right amount of BSs affect the QoS and energy efficiency factor [33]. Another crucial factor is planning and deploying a cell, which is essential for operational cost and capital expenditure (OPEX/CAPEX) perspectives. The goal of 5G and beyond networks is to provide a scalable architecture that provides multiple capabilities using dense networks, but creating a framework for analysing and optimising such a network is a significant challenge [34]. Homogeneous networks are easy to plan as compared to heterogeneous networks. The BSs random deployments in heterogeneous networks can lead to increased interference, increased power consumption, and degradation of QoS [35], [36]. Several approaches are used for BS deployment such as: • Spatial point processes such as the Poisson Point Process (PPP) were used over the years to deploy BS. However, PPP has shown the gap between theoretical and practical deployment, which shows its in-competency for real-time BS deployment [37]. • Non-PPP models such as Poisson hard-core process, Strauss process, Matern cluster process, and Poisson cluster process are considered as better options for real-time environments. Still, these models lack the statistical characteristics to deal with interference challenges [38]. • Hexagonal approach has been used over the years for cellular UDN. Although this approach can help to cover the maximum area with a minimum number of BS but still in the case of multiple users (using the same pilot), the pilot contamination problem is huge [39]. Also, this approach cannot model the capacity, and coverage needs in UDN [40]. • Another approach named as Manhattan type grid was introduced, which is also not sufficient for the dense networks as the inter-cell interference increases with BSs density [41]. Lately, some self backhauling and access links are also increasing, especially its usage along in-band full-duplex as self backhauling small-cell deployment is comparatively low in cost [42]. Planning and deploying a UDN requires careful consideration of small cells. Various cell planning objectives are required to cater for the needs of the network, such as [43] • Traffic: Most of the cell planning and deployment plan is based on user traffic intensity. In previous generations, the geographical division of traffic was adequate. However, with the evolution of multiple services, capacity and data needs, the traffic characteristics also changed. • Prospective location: In theory, a BS can be deployed anywhere, but several prerequisites have to be met before the installation. Choosing a suitable location involves considering the height of buildings, topography, traffic density, and the feasibility of the location.
• Base station modelling: A typical BS is not always applicable to every situation. BS type, antenna height, load capacity, CAPEX/OPEX and transmission power are important for BS modelling. • Signal modeling: A proper mathematical formulation of frequency and signal propagation according to distance is necessary. As mapping out a large area is difficult thus, empirical models are used. Different empirical models are available to plan the deployment of small cells. Empirical models abstract the experimental values to estimate the data for small cells planning. These models can be further tuned according to requirements. These models anticipate reflection, structural hurdles (building, physical topology), signal absorption, and its propagation [43]. Figure 2 shows the stages of network planning that are required for a successful deployment. Pre-planning is essential to estimate the covered area of interest and number of BS to provide enough coverage. The second phase of detailed planning helps point out the actual positions of base stations and other factors like physical obstructions. The final stage which helps to optimise and improve the network is post-planning. This phase helps VOLUME 4, 2016    Small cells have the potential to increase the data rate while maintaining coverage and enhancing spectrum usage.
The accurate density of small cell deployment is of utmost importance to fulfil traffic demands, which should not be too dense or sporadic. BS is the primary wireless network element that consumes the most energy, further categorised as radio-frequency signals power consumption and circuit power consumption. The traditional static cell deployment based network planning mainly focus on peak traffic demands, and implementing the same plan for small cells lead to unnecessary energy wastage [48]. However, deploying the network by estimating the approximate location can predict the energy efficiency and performance of the network by taking account of different traffic distributions [49]. Several UDN development and planning frameworks have been proposed over the years for optimal BS density, such as stochastic geometry [47], heuristic methods [46], [50] and greedy based algorithms [44], [45]. Table 5 shows the summary of BS deployment and optimisation techniques discussed in this section.
In [44], the author proposed a greedy algorithm to solve the microcell base station problem. The author proposed the optimal location selection to increase the energy efficiency of UDN while deploying the required set of BS selected through a greedy algorithm. The proposed algorithm can be implemented in both hexagonal and conventional base station topology. Another similar work has been done in [45] using greedy algorithms to deploy microcells. According to the author, the edges of cells are the most optimal locations to install new cells, increasing spectral efficiency. In [47], the user and BS distribution are studied using the stochastic geometry approach. The proposed algorithm can also help operators to estimate the QoS probability of the deployed BS. In [46], a meta-heuristic approach is used to propose an optimised framework for BS deployment. The proposed work is focused on both capacity and BS deployment. The utilities based on individual BS deployment are proposed as the meta-heuristic approach is an excellent solution for dealing with large data sets' complexities. Another metaheuristic approach used the particle swarm optimisation (PSO) algorithms in [50] for uniform BS deployment. The focus is to reduce the number of deployed BS and reduce power consumption by switching off the unnecessary small BSs. An exciting feature of traffic profile is proposed based on day and night traffic. Monte Carlo simulations were used to prove the results.

B. RESOURCE OPTIMISATION STRATEGIES
In UDN, several cellular users coexist with varying radio access technologies (RAT). In this section, various approaches to improve energy efficiency in the UDN are discussed.

1) Resource allocation
Future networks are all about handling massive amounts of devices accessing the wireless channels simultaneously, resulting in channel overloading. Due to the massive deployment of devices, other challenges like network congestion, resource management, and resource alloca-  tion issues also emerged apart from channel overloading. Moreover, the dense deployment of random BSs causes severe interference and increases the computational complexity and resource management/allocation problem. The resource allocation techniques are equally important to achieve efficient resource management and network utilisation. Effective resource management/allocation determines factors to gauge network performance. Recent years have also seen the importance of energy efficiency as a key performance indicator. It is essential to highlight the fairness of any resource allocation strategy as unfair resource allocation may result in further energy-efficiency degradation. Bandwidth is also considered as a resource of the network, and its fair allocation can lead to a more energy-efficient system [67]. Aside from bandwidth allocation, unfair power and spectrum allocation would also result in energy-efficiency degradation [68]. A fair spectrum allocation is also essential due to the limited amount of spectrum available. A high transmission data rate problem is solved in [56] using deep reinforced learning (DRL). A high mobility network model is considered with dynamic network traffic. Due to the dynamic nature of traffic, the author modelled the channels as innercell and inter-cell states. The author focused on channel capacity and high data rates as incorrectly allocated subframes always affect the network transmission efficiency and throughput. A deep neural network (DNN) is applied to extract features from complex network information.
Using DRL, the agents work based on collected rewards from the environment and use throughput, packet loss and end to end delay as the state. The simulation results show significant improvements of the proposed technique in terms of throughput and packet loss rate.
A sub-carrier allocation problem is discussed in [57], where the author focused on the QoS-based cross-tier cooperation resource allocation approach. The author emphasised improving throughput and QoS by dividing this non-convex problem into sub-problems. Stochastic VOLUME 4, 2016 geometry has been used to study the impact of signal to interference noise ratio (SINR) on different user equipment placed under the macrocell and small cell coverage area. Another work in [63] focused on the resource allocation problem and proposed the solution in a distributive manner using Game theory. In order to optimise the full-duplex system, the resource allocation problem is critical due to interference. The author considered both uplink and downlink channels for the resource allocation problem. The proposed algorithm runs until it reaches Nash equilibrium. Simulation results showed significant performance and a 99% fast convergence rate. In [52], a combination of user association and resource allocation problems is addressed in heterogeneous networks (HetNet) using reinforcement learning. Combining both association and allocation problems makes it further challenging. A double deep Q-network is used to achieve the most optimal user strategy. Apart from resources like sub-carrier and spectrum, transmission power is also a prime contributing factor to network performance. For an ultra-dense network with diverse network requirements in [54], a joint energy-efficient offloading and power allocation scheme is proposed. The author formulated the joint optimisation problem as a Mixed Integer Nonlinear Programming problem (MINLP). A Frank-Wolfe algorithm is introduced to find an optimal solution for offloading. An iterative searching algorithm is developed that works based on the difference between two convex functions to perform the optimal power allocation task. The energy efficiency is highly dependent on the dynamic power consumption of base stations. The allocated power of different channel schemes can be optimised based on their gains, resulting in energy efficiency and QoS and throughput. In [55], the problem of resource allocation using a reinforced learning-based approach for power optimisation in the UDN is presented. A multi-agent Qlearning algorithm-based approach is proposed to efficiently solve the power problem in UDN based on a single macrocell and several small cells. The algorithm is trained consecutively to solve the problem. User equipment is an agent, and a network control centre transmits the power based on training results. The proposed algorithm performed better than traditional existing algorithms because it balances the load and efficiently enhances energy efficiency. In [69], the author proposed a mean-field power allocation algorithm for a UDN. A dynamic stochastic game (DSG) is used to model the power allocation problem. Further, this model is fed into the mean-field game model. The strategic decision of allocating optimal power is made by mean-field game theory. Through simulation, it is proved that the proposed algorithm guarantees QoS and enhanced energy. Another game theory-based approach considering the bankruptcy problem is proposed in [70] to address the resource allocation challenge. Bankruptcy theory is best utilised when the players have similar objective functions [71]. In the proposed work, a small cell network is considered to solve the space allocation problem among several players. The small cells have a limited amount of cache space that require balance distribution to further enhance efficient space utilisation and hence are considered bankrupt company. Internet content providers are considered as players to compete with each other. Considering that small cells have less space than players, a coalition is formed based on player's interests. The solution depends on rationality constraint, which means that players' space size should not be less than the players who have not participated in the alliance. The simulation results show the fair distribution of resources. This technique can also be used for power allocation or bandwidth allocation [72] to improve the energy efficiency of the network. Several work has been done on the resource allocation part on orthogonal frequency division multiple access (OFDMA) [56] [57] [58], non-orthogonal multiple access (NOMA) [60] [61] [62], heterogeneous cloud radio access network (H-CRAN) and other multi-antennas technologies which has been discussed in Table 6.
2) User association / Cell selection Changes in the cell state result in energy consumption which can be significantly high. It becomes challenging to optimise the energy cost during the cell switching process. The cell selection function accepts the connection between user and BS through user association/cell selection and admission control. This cell selection and user association are interchangeable terms used for creating a connection between user and BS to optimise radio resources, improve QoS, enhance energy efficiency or load balancing. The association depends on the user equipment, and their access modes [78]. In the case of a UDN that consists of several macrocells and small cell BSs, the process of user association becomes critically complex because of severe interference and dense coverage. Traditionally, the basic scheme of user association or cell selection based on transmitted power fails here due to variation of MBS and SBS transmitted power. Apart from transmitted power, backhauling is also a considerable aspect for cell association [20], [79]. Some conventional techniques used for the association are based on reference signal received power (RSRP), signal to interference noise ratio (SINR), cell range expansion (CRE) and reference signal received quality (RSRQ). Small cell deployment in UDN is considered to lift the traffic load from macrocells [80]. However, based on the received signal, most of the users associate to macrocells result in affecting the small cell splitting gain and energy efficiency [81]. Additionally, users with high data rate requirements prefer to use small cells. However, this does not guarantee the maximum users association with small cells due to several traffic variations. Hence, an optimal user association scheme is always required to balance the association among small cells for improved energy   [82]. Most of the time, BS are assumed to be connected with unlimited power sources, where energyefficient user association refers to the efficient utilisation of BS power by balancing the load. In these scenarios, when the user selects the BS that has low battery levels based on traffic demands will ultimately result in service degradation [83]. The energy efficiency among macrocells also depends on the service type they are providing, which can be improved by offloading their traffic to small cells [84].
A heuristic algorithm based approach is proposed in [74] on the energy-efficient cell selection process. The purpose of the proposed approach is to minimise energy consumption as well as optimise cell selection. A UDN is considered with uniformly deployed small cells in the coverage area, divided into uniform time slots. As the proposed work is based on the total power consumption of the network, all three costs are included in the power consumption model: base-band cost, transmitter cost, and switching cost. To further solve the problem, they divided the work into two parts: to reduce the switching cost and total energy cost of small cells. The switching strategies are crucial in optimising energy efficiency. The author proposed a centralised user association strategy (CUAS) to attach users to individual small cells based on load possibilities to address the energy issue. Similarly, for the small cell based on/off strategy, the neighbour cell loads based heuristic algorithm is proposed where small cells with a heavier load are less likely to be turned off. The work is focused on the load strategy of the adjacent cells and, through simulations, proved significant results compared to the traditional heuristic searching algorithm (HSA). In [76] stochastic geometry-based analysis is used to improve the UDN via a user association scheme. In [77], a complex problem is formulated by adding NOMA into a UDN. It is further divided into two sub-problems to reduce the complexity level, i.e. cell association and resource allocation. The matching theory was used for the user association to enhance throughput and energyefficient resource optimisation, using channel state information and data rate.
In [75], the author proposed an energy-efficient cell selection and power allocation technique. In the proposed work, a candidate cell is selected based on the user's QoS requirements. A minimum data rate constraint is required to be met by the target cell for selection. A candidate cell selection and joint cell selection and power allocation problem both are presented. For multiple users, the issue of power allocation is solved by the Lagrange method. An iterative algorithm is applied for power allocation, which runs continuously until its convergence point comes. The approach showed an increase in energy efficiency using numerical simulations. Different modelling techniques used for energy efficient user association are summarised in Table 7.

3) Interference management
UDN is deployed to increase system throughput, which also leads to improved spectral efficiency and energy efficiency. Small cell dense deployment of the network significantly increases OPEX/CAPEX, energy consumption and interference issues. These interference issues increase with randomness and more dense small cells base station deployment. The decreasing distance between cells creates more severe co-channel interference, which may deteriorate the network performance and edge users QoS. Unlike conventional wireless networks, the interference problem is more challenging in UDN because of its multi-tier architecture. Besides co-tier and cross-tier interference, inter-cell and intra-cell interference factors also exist in UDN. Macrocells and small cells have the major difference in power, resulting in unavoidable interference challenges. Apart from power, the commonly used VOLUME 4, 2016 [88] due to users and BSs proximity. In multi-tier UDN, sharing the same spectrum and RAT among MBS and SBS are creating more challenging energy and interference management issues [85]. Also, small cells are mostly deployed in an unplanned manner in a dense growing network; hence effective interference management is critically important. Efficient interference management helps minimise the energy consumption while it satisfies the spectral efficiency and QoS. Inter-cell coordination is one way to cope with the interference issue; however, it needs signalling overhead due to the dense deployment of small cells. The distributed control is a viable solution to solve UDN interference [20]. Also, due to the dense deployment of small cells, idle and activation and deactivation of BSs manage the interference problem in neighbouring cells.

FIGURE 3: Interference management in UDN
Interference in UDN can also occur in different domains, and proper coordination among interfering cells is required, which is also shown in Figure 3. Dynamic allocations of orthogonal frequency channels are used to mitigate frequency domain interference, whereas blanking of frames is used for time-domain interference [20]. The transmission power is an effective way which is mainly preferred to mitigate interference issue of UDN [48].
Interference management algorithms have become important as they can minimise energy consumption in UDN to some extent. According to [89], when a substantial interference occurs in the wireless network, small cells must be shifted to an orthogonal frequency band to mitigate the interference issue. In the UDN, the state migration process balances the load that cost extra energy consumption and sometimes interference which is more complicated in HetNets. Game theory, machine learning and other heuristic techniques have become popular techniques to solve the interference problem cooperatively. In [85], a graph-based approach is introduced for interference management. A clustering-based graph is introduced based on SINR. Users without any interference issues are not considered, whereas; points are defined on the downlink of BS for already associated users. Another consideration was made based on resources that each point within the same vertex should have different resources. Based on these considerations, a Q-learning algorithm is used to optimise the interference in the defined graph; also, energy efficiency and re-usability is increased. In [86], an indoor femtocell deployment is considered to overcome the interference management problem in an ultra-dense wireless network. The problem is formulated as many-to-many matching games as each macro-indoor user can connect several small cells. The proposed matching algorithm works based on three utility functions. First, it calculates the list of users. Secondly, it identifies the pairs, and finally, evaluates the matching. This matching algorithm works until it converges to a stable matching. The results ensure stable load balancing and improved energy efficiency. Co-tier and cross-tier interference is increasing tremendously in ultra-dense HetNets. In [87] author proposed an interference management scheme using a combination of graph-based clustering and heuristic algorithms. A clustering algorithm is used to divide users and base stations into separate clusters recurrently. After clustering the users, a heuristic algorithm is proposed to allocate sub-channels. The interference is reduced by ordered successive interference cancellation (OSIC) algorithm. The proposed scheme maximised the system capacity and reduced interference.

4) BS switching
Several researchers agreed upon the efficiency of small cells in terms of system capacity, and with the smaller cell size, BS density increases. Nevertheless, on the other hand, this also affects the energy efficiency of the network, as energy consumption increases with network densification. BS sleeping strategy is one way to minimise energy consumption. According to [98] around 50% of BSs can be switched off without compromising the performance of the network, which can result in 43% energy savings. Wireless networks are designed to provide coverage during peak hours. However, during off-peak hours BSs are not fully utilised, which leads to poor energy efficiency and insufficient spectrum usage [99]. Delay factors are also involved in BS switching on offpeak hours [100]. BS can be switched on/off based on two approaches; fractional switching and complete BS switching. Based on these approaches, the BS can be entirely switched on/off, or a fraction of radio resources can be switched on/off [101]. The design of BS switching involves necessary factors for practical energy-efficient switching which are summarised as follows: BS switching faces, • Cell load & coverage: The cell load which is also discussed above is a necessary factor to be considered for an effective BS switching. Besides the BS's own load, it is necessary to consider the neighboring cell load factor as well. As, the deactivation of one BS can affect the QoS of neighbor BSs. Besides neighbour cell load, some other factors are coverage, frequent handover and transition states. • Traffic profiling: Traffic variation is considered as an acceptable criterion for BS switching; however, periodic BS switching action may not be beneficial in some scenarios, such as during peak hours [102].
These fluctuations also result in extra power or energy consumption. Hence, a dynamic strategy must take account of unnecessary fluctuations for actual network applications. • Interoperatibility between technologies: The BS switching strategies can be incorporated according to the technology used to enhance energy efficiency without affecting the quality of service. For example, in ultra-dense deployment, there may be several UEs communicating through device-to-device communication. In these use cases, UEs can still communicate with BS through another BS; hence, switching off BS in this scenario will not block communication [103].
As more than half of the energy is consumed by BS, switching off the BS has decreased power consumption to some extent. Several approaches have been used in UDN to control the BS power, but it is still highly challenging. As the complexity of BS switching is directly proportional to the number of BSs, the optimisation problem can not be solved in polynomial time [104]. Several researchers have employed heuristic algorithms to solve the BS switching problem, although heuristic algorithms employ exhaustive search and are suitable for small-scale networks. In [74], the author formulated a complex inter programming problem based on joint user association and cell sleeping. Heuristic algorithms are employed to reduce the extra power consumption VOLUME 4, 2016 of the network. Initially, the association is performed based on BS switching probability. Heuristic algorithms are computationally slow; hence, the neighbour load is considered for heuristic algorithm based BS switching. In [90], a unique approach is used for the decision-making process using DRL. A fading channel in the downlink is considered to minimise the total power consumption over the operational time. DRL is considered here as it can perform well in the sequential decision-making processes. A generic UDN scenario is taken consisting of M BS and K users. The author reduced the actions space to help the DRL make quick decision making and avoid computational overhead. After eliminating unnecessary information, the agent resolves the BS sleep strategy. Power is allocated, and a reward function is granted back to the DRL agent. The simulation result shows significant performance improvement when compared to Q-learning in power saving. The total power is reduced in the proposed approach. However, the instantaneous power can also be reduced to achieve energy efficiency but has a downside too. When instantaneous power is excessively reduced, it can affect the optimal sleep decision of time slots because of the time-varying nature of the channel. Another article [91] using neural networks proved optimal solutions for energy saving while switching base stations. The proposed method focused on traffic estimation to further decide on selecting switching base stations. The majority of approaches implement BS switching without considering switching energy. The power consumed at the time of switching BS on/off is also necessary. In [92], researchers focused on minimising the energy consumption of UDN using dynamic BS on/off strategy, including the switching power consumption. A dynamic approach is used to switch the BS and allocate the power among BS, considering the switching state. This approach results in reduced power consumption. Most heuristic algorithms end the algorithm early by estimating the adjacent small cell, resulting in frequent switching and more energy consumption. In [74] centralised user association based strategy to achieve minimum switching cost is considered. Further, it also focuses on the power consumption factor by using the neighbour cells information for BS switching purposes. This proposed technique results in better energy consumption as compared to heuristic algorithms.
Author in [93] used stochastic geometry approach to solve the cell switching issue in UDN. A two-tier UDN network is considered, considering MBS to be on all the time, even when small cells are sleeping. The coverage probabilities are estimated for both macrocells and small cells so to be adjusted to maximise coverage. The power consumption is then reduced on the whole network using coverage probabilities values. Apart from techniques only focusing on BS on/off strategies, it can be implemented in combination with some other energysaving approaches like clustering [95], user association [96] [97] [74], interference management [105] and some other as well. The table 9 summarises BS switching in different communication scenarios based on discussed modelling techniques.

5) Cell zooming
The basic idea of BS switching cannot fully satisfy the growing user traffic needs. On the other hand, deploying several small cells causes severe interference and energy consumption. The surge in traffic and diversification, and mobile mobility patterns prompted the need for cell zooming. Cell zooming is a valuable method of balancing cell loads in this regard. In the UDN cell zooming approach, the BS can adjust its cell size and transmission power according to the varying traffic. Cell zooming can be implemented based on several aspects such as (i) Physical adjustments, (ii) BS cooperation, (iii) Relay (iv) Hetnets based [106]. The adjustment of BS transmission power can benefit the UDN in two ways. One is by increasing the transmission power to improving the effective cell selection/user association as less loaded BS can be switched off [110]. Secondly, reducing the transmission power to adjust cell size and minimise energy consumption [111]. A load-based cell zooming approach is proposed in [106] to improve the energy efficiency of the network. A self-defined loadbalancing algorithm is proposed that gradually minimises the power transmission of small loaded cells. In this way, the load is adjusted among neighbouring small cells, resulting in low power consumption and ultimately enhanced energy efficiency. The cell zooming technique is further categorised into two basic algorithms: static and dynamic. The cell zooming covers both network dynamics and transmits power together. However, when the optimisation of transmit power is performed based on user equipment to reduce the BS power consumption or by ascertaining the height of BS antenna based on received signal strength are considered as approaches used for self-healing in cell zooming [112]. Most of the research work has been done on the two-dimensional Euclidean space, which contradicts the three-dimensional estimators. The author in [107], used the three-dimensional estimators analytical model for the density aware cell zooming to increase the capacity. The presented work is based on outrage probability calculations to adjust the power transmission of BS, resulting in a more efficient network. In another research article [108], a data-driven approach is used to achieve energy efficiency via a cell zooming approach. A metric is introduced, which helps sense the relationship between BS to categorise them according to network statuses. This technique is helpful, especially in extreme mobility scenarios, as it reduces the traffic load uncertainty. The proposed datadriven framework helps to reduce energy consumption according to traffic conditions and assures minor service    [109] introduced the use of the model predictive control (MPC) algorithm in the cell zooming approach to not only reduce the transmitted power but also to minimise the computational load. A summary of the articles as mentioned above is provided in Table 10.

6) Traffic & mobility
Traffic and mobility in UDN play an essential role in ensuring user experience. However, limited spectrum and load are challenging in traffic offloading and mobility management. Mobility management is not only crucial for connectivity, but it also ensures the optimal experience as they advance towards destination [116]. Seamless handover in this regard is also a significant challenge. Initially, the handover mechanism was designed for infrequent handover among MBS. However, with the introduction of small cells, handovers occur more frequently. The varying cell size and unplanned deployment are the primary reasons behind frequent handover triggers. Thus it is crucial to implement effective mobility techniques to achieve smooth connections. Apart from dense deployment and arbitrary cell sizes, mmWave, dual connectivity, and cell aggregation also contribute to the mobility and traffic challenges. In [114], a user mobility prediction based AUtonomous pROactive eneRgy sAving (AURORA) framework has been proposed on the idea of an intelligent framework that predicts user mobility behaviour. Initially, a semi-Markov-based framework is used for mobility prediction. The collected data from this mobility model is used to switch the underutilised small cells intelligently. The current location of the user and handover information at any instance time is used to predict future location. Moreover, landmarks and mobility logs of users are used to estimate directions as well. This work is based on handover traces instead of cell loads to initiate sleep cycles and leverages load balancing among cells to decide cell switching, which allows the QoS to be maintained while improving energy efficiency. Simulations for the performance analysis of the proposed framework showed a 68% and 99% increase in total network energy reduction for both low and high traffic times. The primary issue with stochastic and iterative schemes is making assumptions on statistical behaviours of small cells activities. In this regard, the author in [113] used a non-stochastic bandit theory approach to solve the mobility management problem in the ultradense network. The objective is to resolve the energy efficiency issue by minimising unnecessary handover and energy consumption. Several BS switching strategies have been discussed, but the power optimisation issues are critical for ultra-dense H-CRAN. Author in [64] proposed a handover scheme to minimise the power consumption of SBSs in ultra-dense H-CRAN. A margin-based genetic algorithm has been used to achieve optimal decision levels to minimise power consumption and the ping-pong effect.
Traffic and mobility challenges occur due to the limited spectrum, heavy load and user mobility. For this, traffic offloading is considered an effective way to deal with such challenges. The traffic offloading technique is used to deal with heavy traffic, which works by offloading the traffic of heavy nodes to the lightly loaded nodes. VOLUME 4, 2016 As traffic offloading helps to enhance network capability and utilisation, [115] proposed a traffic offloading technique to enhance network sustainability. A machine learning-based autonomous traffic offloading technique was proposed which uses the feature learning technique to learn the link quality. This link state learning helps the network to maintain a stable state. The proposed feature learning algorithm can learn based on the environment as well. Reinforcement learning (RL) is used for strategy learning. The strategy learner keeps updating the terminal periodically to achieve traffic offloading and load balancing. The proposed feature strategy based algorithm proved better results as compare to traditional offloading algorithms. Table 11 summarises the traffic and mobility work discussed in this section.

C. HARDWARE SOLUTIONS
Traditional approaches in hardware designs use worstcase power provisioning. The power consumption of UDN also depends on the hardware components. Most of the work on UDN assumes that the optical front haul is perfect. However, there are small length optical links that have consequential deployment cost, power consumption and performance [117]. BSs, including RF modules, baseband modules, fronthaul/backhaul links and cloud radio access networks (CRAN), are important aspects of UDN performance. According to [118], due to hardware imperfections, it is assumed that the source point must be equipped with a large number of antennas. In contrast, others have single antennas to reduce the amount of power consumption in the network. A typical antenna that captures radio frequency signals leads to a tradeoff between power consumption and size. According to Huawei [119], a 64T64R 5G AAU consumes 1<1.4<2kW of power for BBU. Radio-frequency design consists of many hardware antenna designs for massive MIMO, which helps to reduce power consumption. Power amplifier also contributes to energy efficiency, and high efficient PA means improved energy efficiency [120]. Another practical innovation in wireless communication is separating the baseband unit (BBU) from the remote radio head (RRH) to reduce the power consumption in RF cable, as BBU is placed within a short range from RRH [121].

D. POPULAR MODELLING TECHNIQUES IN UDN
The emerging wireless networks now support diverse applications such as augmented and virtual reality (AR/VR), military, medical diagnosis, internet of things (IoT), and transportation services. Soon, with the expansion of high data rate communication and QoS, future wireless networks need fast and high-performance modelling techniques to provide better computational results. In this regard, three popular modelling techniques, machine learning, game theory, and other stochastic and heuristic techniques, are discussed below. The reasons behind the focus on these modelling techniques are: • Game theory has been vastly applied to wireless networks problems and has significant advantages in routing and resource allocation problems. Due to its various benefits in a competitive environment, it is deployed in UDN to cope with limited transmission resources. Game theory provides the best results in competitive environments. • In the dynamic UDN, there is a need for selfconfigured, self-optimised, self-healing operations that can significantly adapt to the surrounding environment and make optimised decisions. In this regard, machine learning techniques are a promising tool to learn from system uncertainties, challenges and variations and result in potential solutions. • Recent advantages in ultra-dense wireless networks confirms the need for stochastic geometry, especially in networks where user behaviour is required to be analysed in the vicinity of different cells.
For future UDN, it is necessary to incorporate these modelling techniques to handle mutual impacts and increase network performance. In this regard, machine learning, game theory, heuristic and stochastic geometry approaches can address the growing network challenges and ease the coordination between different densely deployed cells.

1) Machine learning
The future of wireless communication needs to be more intelligent to accomplish tasks that cannot be pre-programmed [122]. Traditionally, functional programming, convex optimisation and game theory have been used to solve the energy efficiency problem of wireless networks. These traditional approaches use analytical solutions based on network parameters. However, in a complex network such as UDN, it becomes challenging to model the network dynamics and obtain optimal solutions. With UDN optimising its energy operations became challenging. With the significant increase of hardware and other parameters such as channel state information (CSI), noise effects and power consumption, it is difficult to adapt the changes for efficient link adaptation [123]. Machine learning-based solutions play a considerable role in problem classification, outcome prediction and exploring solutions for UDN. Machine learning is a considerate emerging solution to the growing demands of the UDN while handling its massive information, resources, demands and characteristics [124]. Machine learning techniques can also be used to extract information that can be helpful to develop future autonomous systems. Machine learning is a subset of artificial intelligence that improves performance by learning from network data. Machine learning techniques have evolved significantly and have become very efficient in image processing, computer visions and medical diagnosis. Researchers emphasise using machine learning to enhance the overall performance in various sectors, including wireless networks as well [125]. Also, keeping in view the growing dense network and traffic variation, machine learning can be a practical approach to cell switching decisions. Machine learning can help in predicting the sleeping patterns of base stations based on learning of network traffic and hence can improve energy efficiency [126]. Apart from BS sleeping, spectrum sensing and resource allocation also affect the energy efficiency of the network. While allocating the appropriate power, the network requires channel information. Traditional allocation schemes are highly iterative, time-consuming and incapable of performing energy-efficient procedures [127]. Machine learning here provides the benefit of improving energy efficiency by utilising previously collected network data. Machine learning is classified into three major categories: (i) Supervised learning, (ii) Unsupervised Learning and (iii) Reinforcement Learning. In supervised learning labelled data set is used where both the input and desired output are provided to the learning agent to find the relationship to predict the unseen input. There is no labelled data set in unsupervised learning, and the algorithms explore the inputs by identifying patterns. In reinforcement learning, the agent is responsible for interacting with the environment and on feedback/rewards, agents generate policies. Apart from the traditional division of machine learning into supervised, unsupervised and reinforcement learning, another vital member is deep learning [128]. Machine learning has proved beneficial to solve traditional programming issues in networking, resource allocation, interference and mobility management [129]. Several conditions need to be checked before incorporating ML [130]: • Problem classification: generally, machine learning is used for regression, classification, Markov decision and clustering problems. • Training data: As machine learning, most of the time, require plenty of data for training, it is necessary to think about whether a massive amount of data can be acquired for training purposes. • Time utilisation: Two important factors come under this category: training time and response time [131]. Some applications like resource allocation and management have critical time utilisation characteristics. In this regard, neural network and reinforcement learning proved to be more time-efficient [132]. • Implementation problems: Machine learning implementation depends on the hardware, algorithm complexity, algorithm processing requirements, data storage and collection problems.
Machine learning is further used for regression, classification and clustering purposes which can be incorporated into energy-efficient solutions. The supervised learning algorithms such as Bayesian theory, K-nearest neighbour and neural networks are used for classification and decision trees, neural networks and support vector algorithms are used for regression purposes. Other unsupervised learning methods like K-mean, Gaussian mixture, hidden Markov model and fuzzy C-means are used for clustering purposes. In this paper, several machine learning-based approaches are discussed based that are applied in UDN to improve energy efficiency. Deep reinforcement learning is used in [52], [56], [90], reinforcement learning in [55], [85], [115], artificial neural network in [91] and semi-hidden Markov method which can be used in both supervised and unsupervised learning is used in [114].
2) Game Theory Game theory has been widely used in wireless communication for cooperation schemes. It helps to model actions that have conflicting outcomes [133]. Game theory and conventional optimisation techniques are sometimes considered the same. However, they are different as no single player have the authority to control the outcome [134]. Game theory has been widely used to solve problems in wireless communication due to its problem analysing nature and ability to describe interactive situations [135]. Growing optimisation constraints of dense networks require schemes that can distinguish rational behaviour and reach the equilibrium in a distributed and controlled manner [136]. Over the years, game theory has been proved beneficial to resolve interference [86], [137], mobility [138], and resource allocation/management [139] problem of wireless networks. In the case of UDN, game theory can attain ideal solutions to large scale wireless network problems that are less complex and have low computational complexity [137]. Game theory is well suited for scenarios involving distributed decision-making without the need for a centralised controller where the devices in the network have conflicting objectives, such as in the case of UDN. Another benefit of using game theory for UDN is its ability to make an optimal decision as a response to the game rather than based on other players strategies.
In the case of UDN, the players will be more as compared to conventional wireless networks. Also, the signalling overhead will be huge, along with rapid spatiotemporal variation. In such scenarios, mean-field game theory is the best strategy to apply as it can take optimal decisions in a distributed manner [137]. Non-cooperative games cannot handle a large number of players [140] however it gives good results in strategic problems and conflicting scenarios. A game is an interaction between players (two or more). A player's own decisions and other players' decisions can influence every player's Payoff. The main elements for defining a game-theoretical problem are: • Players: Players are the independent decision-VOLUME 4, 2016 makers of the game. • Actions: Actions are the specific behaviours within a game that players have to choose. • Payoff: Payoff is the amount or value (specific, exact, increase or decrease) that a player receives that maps a player's actions. • Strategies: The strategies in a game are further divided into pure strategies and mixed strategies. For pure strategies, the value of a game is always the same for every player of that game and determines the action that particular player will do independent of any state [141]. In contrast, a mixed strategy is based on probability. • Solution: Nash equilibrium is one of the most widely used solution concepts. It is considered an important factor in game theory which means a player has no regrets once the decision is made. Nash equilibrium also requires two stages [142]: identifying each player's strategy as a reaction to other players game plan. -when a situation occurs where every player is using its best strategy.
A game theory problem solution depends on the nature of the problem. The types of games are selected based on features. Some commonly used game types are: • Static v.s dynamic games: In static games, each player takes decisions simultaneously, unaware of its surrounding player's decisions. On the other hand, in dynamic games, players can take action repeatedly and at different times. • Cooperative v.s non-cooperative games: Noncooperative games are player-oriented games in which the main focus is on the player's utility instead of the whole game [143]. On the other hand, in cooperative games, players tend to make coalitions to take full advantage of the game [141]. • Zero-sum v.s non-zero-sum games: Zero-sum games always have a loser for every winner, but their total values always remain the same. Non-sum zero games are in contrast to sum zero games as all players benefit from the game.
The game theory approach gives the best results when applied to energy efficiency and security problems in wireless networks because of its decision making power in uncertain situations [144]. Game theory gives the benefit of optimising cell-level performance. Due to its distributed decision-making, potential game theory is suitable for the energy consumption issue of small cell dense networks. Some of the widely used game-based schemes are matching game [86], Stackelberg game [53], [139], coalition game [145], mean-field game [137], [146], nash bargaining [147] and bankruptcy game [?], [72].

3) Optimisation & Heuristic algorithms
Apart from machine learning and game theory, other heuristics and stochastic geometry techniques have proved beneficial to wireless networks for improved energy efficiency. Stochastic geometry refers to study the of random spatial patterns. As a field of applied probability, it is used to estimate spatial averages on higher dimensions. Point processes are inherent of stochastic geometry [148]. The PPP model is an efficient way to deal with wireless network interference. Power consumption and interference are interlinked with each other as interference can be minimised by adjusting the power consumption of BS, but that affects the downlink and uplink coverage [149]. Another way to enhance energy efficiency and minimise interference is to use stochastic geometrybased PPP, where the point processes are considered random points and are spread over space. Apart from joint power consumption and interference perspective, stochastic geometry is efficiently used in resource allocation to improve EE because of the complexity caused by many access points and users in UDN. Besides the computational complexity of UDN, complete network information is required to gain the full benefits of dense networks. Also, the large number of access points and complexity leads the network towards extra energy consumption. Stochastic geometry has the benefit of solving high complexity computational problems, which eases the computational procedures [150]. Heuristic algorithms are best for the situation where fast results are required. Heuristic algorithms can prove to be energy-efficient in network management as it requires minimal resources for computation and maintains user QoS. According to [151], the heuristic algorithm is further divided into availability, representativeness, anchoring and adjustment. The heuristic approach is best utilised when memory or data is available to call the statistics. It is best utilised in energy-efficient resource management problems. The representative heuristic approach makes decisions based on probability or random likelihood. The anchoring and adjustment heuristic gives an initial value that further increases or decreases as per estimations. The initial decision part is known as anchoring; most of the time, the algorithm is stuck at this point, resulting in biased results. The major benefit of the heuristic algorithm is its immediate automatic response which makes it more efficient. Stochastic and other heuristic algorithms like greedy based algorithm [152] and genetic algorithm [153], [154] are proved beneficial in solving energy-efficient resource allocation, cell sleeping [94], user association [155] and scheduling [156] problems in UDN.

E. ENERGY HARVESTING
Besides the excessive demand for speed, capacity, and coverage, the ICT industry should also consider economic and environmental issues. Due to the annual increase in network operations costs, average revenue per unit (ARPU) decreases annually. One of the significant factors of these increasing OPEX/CAPEX rates and environmental footprint is the energy consumption to maintain a network [157]. Researchers are looking for efficient ways to harvest energy through environmental sources (thermal, solar, wind) and hardware like BS, sensors and other mobile terminals to provide eco-friendly and energysaving networks. The obtained energy from renewable energy sources can be further stored and utilised directly. This section discusses three significant ways to harvest energy for wireless networks: natural energy harvesting, coupling techniques, and wireless power transfer (WPT).

1) Wireless Power Transfer (WPT):
WPT is a general method to transfer power across devices that have been used over the decade. Generally, the power transfer range is from some microwatts to milliwatts. However, the industry is now working on high power applications to increase the range to several kilowatts [99].
• Wireless Energy Transfer (WET) concentrates on the downlink side of the wireless network. It transfers the energy from the base station to user equipment.

• In Simultaneous Wireless Information and Power
Transfer (SWIPT), energy and information both are transferred simultaneously in downlink [158]. • Wireless Powered Communication Network (WPCN) works the same as SWIPT, but both uplink and downlink are used to transfer information and energy, respectively [159].
2) Natural energy harvesting: Energy is harvested through natural resources abundantly available in the world like sun, water and wind. However, these kinds of harvested power are not consistent due to the availability of the sources. 3) Coupling techniques: Inductive and magnetic coupling are two methods to harvest short-range energy.
F. PERFORMANCE MEASURES P.1 Energy efficiency: In simple terms, energy efficiency is maximising the number of bits transferred per unit of energy. In UDN, deploying many devices makes the network consume more energy, which is a significant cause of concern. Also, to minimise the OPEX, it is necessary to keep energy consumption low. Therefore, energy efficiency schemes are of utmost importance for 5G. P.2 Throughput: Throughput is how much actual amount of data is received or forwarded across a communication link. Throughput can be measured in both bits/sec and data/sec. Not only is throughput an effective way to measure network performance, but it can also be optimised to minimise latency. The bandwidth is different from throughput because of latency, packet loss, jitters, and other parameters. P.3 QoS: Quality of service (QoS) is used to measure the overall network performance by users. Considering the growth in current and future networks, meeting quality of service has become more critical. Speed, end-to-end delay, reliability, jitters and bandwidth all aspects are all covered under QoS. It also includes handling and controlling network resources and prioritising them for different data types. Enhancing QoS is one of the critical objectives of an energyefficient ultra-dense network as heterogeneous devices in UDN have different types of data requirements and priorities. P.4 Data rate: The data rate is defined as the number of bits transferred per second time. The data rate is the transmission speed and is denoted by bytes per second. P.5 Fairness: The sharing and allocation of resources become a challenging task in UDN because of the dense deployment. It is measured as the difference between the highest and lowest values of any network parameter. Fairness is mainly related to the resource allocation aspect of wireless networks. The fair allocation of resources has become a critical issue that needs to be addressed. As the unfair allocation of resources may lead to lack, wastage or surplus of resources. Fairness can be estimated as quantitative (Jain's index and entropy measure) and qualitative measure (max-min and proportional fairness). P.6 Packet loss rate: The packet loss is the percentage of lost packets compared to the total number of packets sent. This packet loss occurs mainly because of poor network, multi-path fading, interference, network congestion or errors during transmission. Extreme packet loss also affects the QoS and throughput of the network. P.7 Total power consumption: Power consumption defines the power consumed per unit time to operate the equipment. It is measured in kilowatts (kW) or watts (W). In wireless networks, some energy is also wasted as electromagnetic radiation, heat or vibration. P.8 Sum rate: The total amount of communication taking place in a network defines the sum rate. Sum rate is directly proportional to data; the more significant the node communication, the more data can be transferred. P.9 Coverage: The geographical area covered by the BS to provide communication facilities to the user equipment is known as the coverage area of that particular BS. Reudink's formula is used to calculate the relationship between coverage at edges of BS and cell area coverage probability. VOLUME 4, 2016 Energy efficiency metrics play an important role in energy-efficient communication as it gives quantified information [160]. Energy efficiency metrics are applied to: • compare the performance of energy consumption among different components • direct future energy efficiency research and development goals • emphasises adapting more energy-efficient configurations in the network There are different metrics to evaluate the energy consumption of wireless networks. Energy efficiency metrics are used to either measure the performance-based energy consumption or the energy efficiency improvement [161]. According to the requirements, energy efficiency metrics can be further divided into three major classes:  [160].

IV. CHALLENGES AND FUTURE RESEARCH DIRECTIONS
With the proliferation of devices, sensors and datahungry applications, energy-efficient technologies will be necessary. There has been much investigation done on energy-efficient techniques in UDN. However, there are several challenges and future research directions that need further investigation, as summarised below: 1) UDN and big-data convergence: 5G is envisioned to increase the network capacity; however, due to the design philosophy of UDN, it results in increased energy consumption. This problem becomes more severe when big-data technologies are combined with UDN, especially with the deployment of many devices, as in IoT. More power is required to operate an extensive deployed small cell network because of differences among temporal and spatial traffic loads. In the future, more work can be done to improve energy efficiency by utilising big data to assist the deployed UDN. Also, artificial intelligence can be utilised for big-data-based UDN to deploy it practically without incurring increased energy consumption. 2) System analysis in UDN: The system analysis is essential to identify the challenges that need to be addressed appropriately. In the case of UDN, increasing dense networks are deployed to satisfy users coverage and capacity issues. However, practically analysing the dense system becomes challenging. There are network deficiencies in every wireless network that cannot be ignored. Like in UDN, the impact of shadow fading is more critical because of the dense deployment, which affects network performance and causes energy wastage. More work can be done on the system analysis part, which will help start new research areas in UDN networks and energy efficiency. 3) Energy harvesting & traffic patterns: Though researchers have started working on WPT to supply power to network elements, much work is required to address the small power efficiencies. Focusing on network traffic patterns and utilising energy cooperation techniques efficiently by accessing those patterns can benefit the network. 4) Overhead information exchange: Information exchange among network components requires energy to communicate. With the increase of devices and traffic in UDN, information exchanges among network components also increased exponentially, resulting in more energy consumption. Apart from increased energy consumption, it also causes an interruption in network outage execution. Hence, future work is required to analyse and evaluate overhead information exchange and its impact on dense small cell networks. 5) Vertical densification: Although vertical densification provides more benefits than horizontal densification, it is a challenging task to do vertical densification of small cells. Vertical densification is modelling SC base stations in an elevated plane, which is not easy in UDN. More research work can be done in future on the performance evaluation, modelling, frequency reuse techniques and energy efficiency problems of UDN small cell vertical densification. 6) Lack of training data: Machine learning has been studied widely in many applications and decisionmaking processes from massive data. However, there is a lack of data sets for research purposes in the wireless network domain. Apart from data sets, the security risk is also associated with these big data sets used for training complex and high-performance models. For the future, there is a space to work on the machine learning model to train huge data sets, especially for improving energy efficiency scenarios. 7) Energy efficiency of machine Learning: One of the most significant benefits of machine learning is its iterative learning nature from the environment. Data acquisition, training, testing, validation, and debugging consume a considerable amount of processing. As there is a strong correlation between energy consumption and time, it is vital to work on the energy efficiency of machine learning itself. Reducing the energy consumption of machine learning can help to reduce the energy consumption of networks that have machine learning implemented in them. More work is required to further work on machine learning-based scenarios to enhance energy efficiency further. 8) Security issues: Wireless networks are always more susceptible to security issues. These non-negligible security challenges are crucial for small cell dense networks as they compromise integrity, trust, authentication and energy consumption. Two main approaches are generally used to secure the network: upper-layer encryption techniques and securing the physical layer [164]. The major drawback of these encryption techniques is the computational cost and energy to defend these attacks that overload small cell base stations. Other security concerns of UDN are handover management attacks and cell interference attacks in UDN, where attackers intentionally increase the number of small cells in the network to create confusion among users. Mitigating these attacks require a lot of energy consumption in terms of processing. Future work must work on the attack defence schemes that reduce the complexity and minimise power consumption during this mitigating phase.

V. CONCLUSION
There are several methods of increasing wireless network capacity, but the most promising is ultra densification. UDN is a promising technology to increase the capacity and performance of future networks. Beyond the capacity and data aspect, energy efficiency is a critical factor for future wireless networks. One way to control energy consumption is by adjusting the power consumption of BS while satisfying the coverage and performance criteria for users. There are, however, other ways to increase the energy efficiency of UDN. This article provides a detailed survey of energy-efficient small cell networks, along with different research directions to improve the energy efficiency of UDN. Based on the review, we have provided the detailed taxonomy of UDN that highlights the popular approaches, modelling techniques, performance measures, and energy efficiency metrics. The discussed approaches cover the UDN from network planning & architecture, resource optimisation, hardware solutions, and energy harvesting. Further, resource optimisation strategies consist of resource allocation, user association, interference management, BS switching, and cell zooming schemes to address the energy efficiency issues in UDN. The latest modelling techniques (machine learning, heuristic algorithms, stochastic geometry, and game theory) and their roles in UDN are discussed to improve energy efficiency. Besides energy efficiency, the discussed approaches also focus on reducing interference, improving QoS, and enhancing spectral efficiency, directly and indirectly affecting small cell UDN. In addition to discussing the advantages of UDN, we outline a few challenges that require more in-depth research and present future research directions. We believe that the proposed taxonomy will help future research directions and motivate potential research to improve energy efficiency in UDN.

MUHAMMAD AMAN SHEIKH is Lecturer at
School of Science and Technology, Sunway University. Previously he was a Senior Scientist in the RD sector and was involved in enormous projects for PETRONAS. He is a Fellow of the Institute of Electrical and Electronic Engineers, Malaysia (MIEEE) and since 2010, a Registered Engineer with the Board of Engineers. An active researcher, he has published extensively and established global collaborations by working with international collaborators and securing international grants. He was awarded the Graduate Assistant Scholarship during his MSc and PhD in Electrical and Electronics Engineering in 2013 and 2018 respectively. He also received prestigious awards during his PhD candidature including Best Graduate Assistant Award and High Achiever Award.
ABDUL AHAD received the M.Sc degree in computer science from Swat University, Pakistan in 2014, the MS degree in computer science from Virtual University, Pakistan in 2017. He is currently pursuing a PhD degree in computer science with the Sunway University, Malaysia. His research interests include 5G, Internet of Things, wireless networks, Wireless Body Area Network (WBAN) and artificial intelligence. He has the author of many peerreviewed research articles that have been published at various top conferences and journals. He is a regular reviewer of many journals including IEEE Access, International Journal of Distributed Sensor Networks, Sensors and others.