Deep Learning for Resource Management in Internet of Things Networks: A Bibliometric Analysis and Comprehensive Review

In this study, we conducted a bibliometric analysis and comprehensive review of the studies published between the period of 2012 and 2022 on resource management in internet of things (IoT) networks using the Scopus database to determine the current state of research and gain insight into the research challenges and opportunities in the field. The bibliometric analysis technique was employed to bibliometrically analyze the published studies that were collected from the Scopus database and this helped to discover the majority of research subjects in the field of resource management in IoT networks. Following this, we conducted a comprehensive review of the relevant studies to provide an insight into the recent progress and the research gaps in the field. According to the results of our bibliometric analysis and the comprehensive review, we discovered that resource management problems in IoT networks is still a growing challenge as a result of the limited available resources for operating IoT networks. Resource management problem is a critical research area due to the advantages of IoT in terms of collecting vital data that could be used in analyzing and predicting human behavior as well as environmental conditions. Also, the results of our bibliometric analysis and comprehensive review further revealed that research on the use of conventional artificial intelligence techniques, such as optimization approaches and game theory approaches, for resource management are common, while research on the use of the modern artificial intelligence technique, like deep learning approaches, is less common. Therefore, this study aims to fill the research gap in the area of resource management in IoT networks by introducing the use of deep learning approaches. Deep learning is a powerful artificial intelligence method that is advantageous for obtaining low-complexity resource allocation solutions in a near real-time. Also, various open research issues that are associated with the use of deep learning approaches are highlighted as future research directions to enable the development of novel deep learning models for IoT networks.

ABSTRACT In this study, we conducted a bibliometric analysis and comprehensive review of the studies published between the period of 2012 and 2022 on resource management in internet of things (IoT) networks using the Scopus database to determine the current state of research and gain insight into the research challenges and opportunities in the field. The bibliometric analysis technique was employed to bibliometrically analyze the published studies that were collected from the Scopus database and this helped to discover the majority of research subjects in the field of resource management in IoT networks. Following this, we conducted a comprehensive review of the relevant studies to provide an insight into the recent progress and the research gaps in the field. According to the results of our bibliometric analysis and the comprehensive review, we discovered that resource management problems in IoT networks is still a growing challenge as a result of the limited available resources for operating IoT networks. Resource management problem is a critical research area due to the advantages of IoT in terms of collecting vital data that could be used in analyzing and predicting human behavior as well as environmental conditions. Also, the results of our bibliometric analysis and comprehensive review further revealed that research on the use of conventional artificial intelligence techniques, such as optimization approaches and game theory approaches, for resource management are common, while research on the use of the modern artificial intelligence technique, like deep learning approaches, is less common. Therefore, this study aims to fill the research gap in the area of resource management in IoT networks by introducing the use of deep learning approaches. Deep learning is a powerful artificial intelligence method that is advantageous for obtaining low-complexity resource allocation solutions in a near real-time. Also, various open research issues that are associated with the use of deep learning approaches are highlighted as future research directions to enable the development of novel deep learning models for IoT networks. 22 INDEX TERMS Internet of Things, resource management, resource allocation, artificial intelligence, game theory, optimization theory, machine learning, deep learning, bibliometric analysis.
to assist researchers who are interested in this research area. 95 Also, this paper motivates the use of deep learning approaches 96 for solving major resource allocation problems in the IoT net-97 works to improve on the computational complexity problems 98 of the optimization theory and game theory approaches. Deep 99 learning is a powerful modern artificial intelligence method 100 that is advantageous for obtaining low-complexity resource 101 allocation solutions compared to other artificial intelligence 102 methods such as optimization, machine learning (ML), and 103 game theory. Moreover, this paper is closed with the pre-104 sentation of some research challenges and future research 105 directions to develop new sophisticated resource management 106 algorithms for IoT networks using deep learning. Following 107 these efforts, the major contributions of this paper are pre-108 sented as follows: 109 1) We provide a bibliometric analysis of the studies 110 published on resource management in IoT networks 111 between the period of 2012 and 2022. 112 2) We provide a comprehensive review of optimization, 113 deep learning, and game theory approaches in wireless 114 IoT networks, along with their benefits and disadvan-115 tages. We also provide a comprehensive review and 116 analysis of the resource allocation solutions that are 117 based on game theory, optimization, and deep learning 118 approaches for IoT networks. 119 3) The performance comparison of resource allocation 120 solutions using deep learning theory, game theory, and 121 optimization theory approaches in IoT networks was 122 presented. 123 4) The provision of future research directions for develop- 124 ing novel resource allocation approaches for IoT net- 125 works based on the promises inherent in deep learning. 126 The details about the structuring of this work are pro-127 vided as follows. Section II presents the research design and 128 methodology of this study. Section III presents a discussion 129 on the benefits of IoT networks and resource management 130 challenges associated with IoT networks. Section IV presents 131 a review of key optimization approaches that could be used 132 to seek solutions to resource management challenges in IoT 133 networks. Section V presents a review on the basics and 134 use of deep learning to improve the resource management 135 challenges in IoT networks. Section VI presents a review 136 on the examples of the game theory approaches used for 137 solving resource management challenges in IoT networks. 138 In Section VII, the comparison of game theory, deep learning, 139 and optimization theory approaches is presented. Section VIII 140 presents the major challenges associated with the use of deep 141 learning approaches for solving resource management prob-142 lems in IoT networks and the key future research directions. 143 Section IX concludes this study.   research gap provides a scope for more research efforts on the 211 use of deep learning approaches to improve resource man-212 agement in IoT networks. The blue cluster denotes ''cloud 213 computing''. The blue cluster is strongly linked with ''fog 214 computing'', ''computing resource'', ''fog node'', ''mobile 215 device'', ''genetic algorithm'', and ''fog computing environ-216 ment''. The cluster revealed that research interests in the 217 use of computing technologies for resource management in 218 IoT are less popular due to the complexity of allocating the 219 computing resources of such technologies to the IoT devices. 220 The blue cluster further revealed the use of optimization 221 approaches, like genetic algorithm, to compute the alloca-222 tion of the computing resources of fog computing. It is also 223 important to point out that the clusters are strongly linked 224 with ''management''. This is an indication that ''manage-225 ment'' is a popular and leading research in the published 226 studies over the last decade. Additionally, this reveals that 227 ''management'' is a major research area in IoT towards 228 addressing the resource management challenges associated 229 with IoT. indicates that within the timeline, more studies focus on the 246 use of optimization approaches for resource management in 247 IoT networks while some studies also focus on the use of 248 game theory approaches for resource management in IoT 249 networks.

250
Therefore, according to the results of our bibliometric 251 analysis of the studies published between the period of 252 2012 and 2022, resource management in IoT networks is 253 still a growing challenge as a result of the limited avail-254 able resources for operating IoT networks. Consequently, 255 the resource management problem is a critical research area 256 due to the advantages of IoT in the context of collecting 257 vital data that are useful for analyzing and predicting human 258 behavior as well as environmental conditions related to air 259 quality, water quality, and weather. Also, the results of our 260 bibliometric analysis revealed that research on the use of 261 artificial techniques, such as optimization approaches and 262 game theory approaches, for resource management are com-263 mon while research on the use of artificial intelligence, 264 like deep learning approaches, is less common. Based on 265 the results of the bibliometric analysis, we were able to 266  broker. The data publisher represents the data source, the 327 data broker performs the role of receiving data for a topic 328 published by the publisher while the data consumer performs 329 the role of subscribing to the topics managed from the broker. 330 Some examples of the publish-subscribe protocols include 331 AMQP and MQTT. The request-response model provides a 332 stateless bidirectional communication setup between a client 333 and a server where the client sends a request to the server and 334 the server provides a response to the request. Some examples 335 of the request-response communication protocols include 336 XMPP and RESTful HTTP. The push-pull communication 337 model entails a data publisher that pushes its data into a data 338 queue and a data consumer that pulls the published data from 339 the data queue. An example includes a queue-based protocol. 340 The use of IoT technology is vital to everyday human 341 activities. Such activities can be classified into several areas, 342 including smart industries, smart environment, smart city, and 343 smart health [1], [21].

368
For instance, in smart water quality sensing and water 369 supply applications, IoT technology could assist to efficiently 370 monitor changes in water quality, control the distribution 371 of clean water for various consumption uses, guarantee the 372 safety of the public health since it helps to increase access to 373 clean water, and prevents the distribution of unclean water to 374 the public.

375
In smart industries, under the concept of Industry 4.0, IoT 376 technology could be leveraged to monitor and manage several 377 industrial applications and processes by connecting machines 378 that combine different sensor devices to a central system to 379 allow visualization and decision activities.     while sending their critical data. This is due to the nature of 434 the data of critical applications and the need to ensure the 435 safety of lives. As a result, critical application data needs 436 to be communicated timeously to the appropriate quarters to 437 aid quick decisions. To support the data latency requirements 438 of different devices with low computational power, effi-439 cient computational resource management algorithms must 440 be developed to improve the latency performance in critical 441 IoT applications. The devices deployed in various critical IoT applications are 444 mostly heterogeneous in nature due to the increasing use of 445 different detected parameters in sensor fusion applications. 446 This results in heterogeneous traffic with different throughput 447 requirements. Bandwidth is a scarce resource in IoT appli-448 cations due to limited available electromagnetic spectrum, 449 but further depends on the transmit power as another scarce 450 resource. The objective of bandwidth resource management 451 is to increase the achievable throughput of the IoT devices 452 in the IoT networks to improve their data transmission per-453 formance. Since bandwidth determines the data transmission 454 capacity (i.e., throughput) of a wireless channel according to 455 Shannon's equation [82], [83], efficient bandwidth resource 456 management algorithms will contribute to supporting the 457 throughput requirements of different devices. The channel is a communication medium that is used by the 460 devices in an IoT network to exchange control messages and 461 packets in the downlink channel and uplink channel, respec-462 tively. The control message from a base station device is used 463 to schedule the IoT sensor devices to transmit their packets 464 to the base station over the uplink channel [84]. Due to the 465 limited channel resource and the quantum number of IoT 466 devices that want to use the channel, the objective of chan-467 nel resource management is to prioritize control messages 468 as well as manage (control) the channel. Channel resource 469 management can be achieved by formulating the channel 470 resources as a resource allocation problem and solve using 471 different artificial intelligence techniques. Furthermore, the 472 IoT networks may integrate various devices that wants to 473 sense and communicate critical data to the base station. As a 474 consequence, for proper utilization of the channel it is very 475 important to design different access management schemes 476 to manage the devices channel utilization process to pre-477 vent problems like access collision, energy wastage, energy 478 consumption, and delay. For example, it is well established 479 that significant energy is mostly expended by IoT devices 480 during data communication due to several factors, includ-481 ing the wireless channel conditions causing congestions and 482 collisions. Hence, another objective of the channel resource 483 management is to manage how the IoT devices can efficiently 484 access the channel.    could be established. When convex optimization approaches 520 are employed to solve IoT networks resource management 521 problems, optimal solutions are typically obtained to such 522 problems.

523
In literature, convex optimization approaches have been 524 developed for solving resource management problems related 525 to wireless IoT networks. A good example is presented in [85] 526 where an interior method-based resource allocation algorithm 527 was proposed to jointly solve power and transmission time 528 allocation problems in IIoT to compute optimal power and 529 transmission time solutions for improving user fairness and 530 throughput.

531
Advantages: Convex optimization approaches are suitable 532 for obtaining an optimal resource allocation solution for IoT 533 applications resource management problems. Heuristics are problem-specific techniques that have been 541 widely employed in wireless IoT networks, either separately 542 or jointly with other optimization techniques, to solve com-543 plex resource management problems when other optimization 544 techniques do not fit.

545
Heuristic optimization approaches in wireless IoT net-546 works may be developed using the optimization framework 547 of problem-independent metaheuristic algorithms or logical 548 ideas depending on the resource management problem that is 549 formulated in the context of complexity.   make the reward function converge quickly to a near optimal 606 solution for an objective function.

607
For example, in [90] the authors formulated a non-convex 608 energy efficiency optimization problem owing to the lack 609 of convexity of the problem structure. To address the power 610 and time management issues of the formulated optimization 611 problem, a PSO algorithm was adapted. In [91], we describe 612 an adapted PSO algorithm to solve both time and power 613 resource management problems of an IoT network to improve 614 energy efficiency. In [92], a PSO algorithm was applied to 615 a cognitive wireless sensor network to address the spectrum 616 sensing problem and determine which of the devices that 617 may use the channel in order to improve the network energy 618 efficiency and throughput. In [93], the authors considered 619 the application of ACO to the problem of computational 620 overhead in IIoT to compute near-optimal solutions that can 621 reduce the computation overhead and latency to increase 622 the efficiency of the system. The authors of [94] developed 623 a forest optimization resource allocation algorithm for the 624 proposed IoT system to reduce the energy consumption and 625 delay associated with the process of computing and allocating 626 resources. The work in [94] also considered other conven-627 tional optimization approaches that are based on GA and 628 PSO. The proposed forest algorithm was compared with both 629 GA and PSO, and the forest optimization resource allocation 630 algorithm outperformed the other algorithms in terms of com-631 putational complexity and network performance.

632
Advantages: Meta-heuristic approaches work well for the 633 IoT network resource management problems they are applied 634 to and may be adapted to most IoT resource management 635 optimization problems in practice.

636
Disadvantages: The computation of resource allocation 637 decisions for obtaining solutions to resource management 638 problems using meta-heuristic algorithms require intensive 639 online computations that expends the scarce power resources. 640 Also, in practice, resource allocation meta-heuristic algo-641 rithms are computationally complex and costly because they 642 incur high timing overhead during operation, especially when 643 many IoT devices are considered. Unfortunately, the IoT net-644 works in time-critical applications may not tolerate the delay 645 due to the timing overhead and computational complexities as 646 such applications require a real-time processing and are sen-647 sitive to delay due to their critical data to human lives, public 648 safety, health, and well-being. Solutions obtained to most IoT 649 network resource management problems in literature using 650 meta-heuristic algorithms are only near optimal, which may 651 obviously impact the QoS performance of the network. This 652 limitation is typically due to the settings of parameters and 653 operators for the designed objective functions to be solved. 654

655
A summary of the reviewed optimization method is presented 656 in Table 2

676
DL is a data-driven approach that leverages data to solve 677 resource management problems in practical IoT networks.

678
As described in Figure 5, DL also uses a processing pipeline 679 that is similar to ML, but then, DL uses a generic feature  The input, hidden, and output layers are densely connected 732 layers of a deep learning model, and each layer may consist 733 of multiple neurons. The input layer is used to only accept 734 and pass the input data x (e.g., network data such as channel 735 realizations) to the hidden layers positioned at the centre of a 736 deep learning network. No computation is performed by the 737 neurons in the input layer.

738
The hidden layer is used to perform computations like 739 feature extraction, transformation, weighted sum, and non-740 linearity of the weighted sum on the input data through 741 its neurons. For example, each neuron of the hidden layer 742 does a non-linear operation on the input data. Each neuron 743 computes the weighted sum ( ) or net input h of all its 744 input data by multiplying each signal with its corresponding 745 weight and adding up the computed dot products, and sending 746 the weighted sum to its activation function as described in 747 Figure 6 and Eqn. (1) [99].
where x is the input data and w is the weight of the connection 750 link between the neurons in the input layer and the neurons 751 in the hidden layer.

752
An activation function is a mathematical function that 753 enables the neurons in a DNN to communicate with each 754 other over their weighted connections. It converts the 755 weighted sum to a linear function as an output y. This output 756 is then passed to the next layer through another associated 757 weighted connection. The examples of some available acti-758 vation functions in DL/DNN are sigmoid function, rectified 759 linear unit (ReLU) function, and tanh function. The sigmoid 760 function takes real number values as an input and convert it 761 to an output that is restricted to a value between 0 and 1. 762 The sigmoid function produces an s-shaped curve. The ReLU 763 function converts the input of whole number values to an 764 output of positive numbers, and produces a rectified curve. 765 While the tanh function also takes real number values and 766 VOLUME 10, 2022 and 1 [99]. Similar to the sigmoid function, the tanh function 768 also produces an s-shaped curve. Table 3 presents the math-769 ematical representation of these commonly used activation 770 functions.

771
Using the sigmoid activation function in Table 3, for exam-772 ple, the computation of the output value y of the neuron given 773 in Figure 6 is described in (2) and (3) [99] as: The ReLU function is advantageous for performing com- e is the expected output and y (k) p is the predicted 815 output of the kth training sample for a given input sample x (k) . 816 From (5), the cross entropy loss uses negative log probabil-817 ities to find the difference between the predicted output and 818 the expected output.

819
The loss function represents the cross-entropy loss between 820 the expected output and the predicted output or the measure 821 of the prediction error of a model. The loss function is used to 822 determine if the prediction accuracy of the trained DL model 823 is good. For example, the lower the loss function, the higher 824 the prediction accuracy of the trained model.  The use of DL to solve resource management problems in 843 IoT applications relies on training and building a DL/DNN 844 model to which network data in the form of channel rep-845 resentation or matrix representation is provided as training 846 sample inputs. This requires following the DL model building 847 pipeline described in Figure 5 to train and test a DL-based 848 resource management model in any of Tensorflow [107], 849 MXNet [108] or PyTorch environment [109]. The model 850 is then evaluated to investigate its prediction accuracy by 851 testing it on unseen channel data samples that it has not been 852 exposed to before. The result visualization of the model is 853 also carry out to visualize the results of the model using the 854 Matplotlib tool [110]. The model deployment phase is used 855 to deploy the DL-based resource management model that 856 have been trained and tested in a Keras, Tensorflow,MXnet,857 or PyTorch environment where it can be compiled into an 858 executable form for deployment and exported to different IoT 859 devices hardware/processor platforms like the Texas Instru-860 ments [111], Intel [112], ARM [113], and Raspberry Pi [114]. 861 Figure 8 gives an insight into the process of DL model 862 compilation and deployment on IoT devices.

863
The trained model may then be used to compute and pro-864 vide a resource allocation solution to a resource management 865 problem. However, the computation of the resource allocation 866 solutions may be intensive as each layer of the model carries 867 out the task of matrix-vector multiplications [115]. But then, 868 it may be advantageous over the conventional optimization 869 approaches depending on the design.   To make the reading of this paper to be interesting, a list of 871 the abbreviations of some important terms used in this section 872 is presented in Table 4. ings. This must be considered when selecting a DL algo-883 rithm for designing a DL-based resource allocation algorithm 884 for IoT networks. An example of a DL model for solving 885 resource management problems (e.g., time resource alloca-886 tion and power resource allocation) in IoT applications is 887 given in Figure 9. In Figure 9, we show how the input data in 888 the form of a channel representation or matrix representation 889 is fed into a DL/DNN architecture through the input layer to 890 predict power resource allocation for the IoT devices in the 891 IoT applications. 892 VOLUME 10, 2022      [117], [118], and [119].

928
In [117], a supervised DL based approach was presented 929 to predict an optimal transmit power for different channel 930 coefficients in a wireless powered communication network 931 (WPCN). The authors employed a multilayer perceptron 932 (MLP) architecture to learn the resource allocation solutions 933 (i.e., output labels) of an iterative optimization algorithm used 934 to solve the formulated transmit power minimization problem 935 and the channel vectors (i.e., input labels) that correspond 936 to the resource allocation solutions as the training data. The 937 proposed model achieved an approximate resource alloca-938 tion accuracy compared to the iterative optimization algo-939 rithm using the standard MSE for performance evaluation. 940 The authors did not report the percentage of the prediction 941 accuracy for the proposed model. Also, the proposed model 942 achieved an improved computational complexity against the 943 baseline iterative optimization algorithm.   In [118], a supervised DL based approach was pre-952 sented to predict the optimal transmit power and PS ratios 953 resource allocation that can minimize the sum-transmit-954 power of a SWIPT-based IoT system. They used a con-955 ventional optimization algorithm to solve the optimization 956 problem of the paper. The power and PS ratios resource 957 allocation solutions (i.e., output labels) computed by the 958 optimization algorithm with their correspondence channel 959 vectors (i.e., input labels) were learned by using four DL 960    the channel gain in order to maximize the energy efficiency 1017 or the spectral efficiency of the network. In the paper, the 1018 channel samples (in dB) of a pre-trained CNN was used 1019 to reproduce an existing power control scheme of transmit 1020 power results for the given channel data samples. Also, the 1021 authors used a CNN architecture with a 3 X 3 convolution 1022 to perform a 2D spatial convolution on the input data. The 1023 channel samples (i.e., the training data) are fed into the CNN 1024 model to find a transit power for each channel sample and 1025 to train a CNN model. Then, the model was used to pre-1026 dict an optimal transmit power allocation based on current 1027 channel state information to improve the energy efficiency or 1028 the spectral efficiency of the network. The performance of 1029 the proposed model outperformed a baseline CNN model in 1030 terms of computational time. The authors did not report the 1031 percentage of prediction accuracy. The developed model was 1032 deployed on the system devices by testing it in a deployable 1033 environment.

1040
The unsupervised deep learning approach is formed by com-   Advantages: The proposed DBN model for resource allo-1079 cation was able to achieve a near real-time computational 1080 time for resource allocation to the devices in the system.

1081
Disadvantages: The computation power of the proposed 1082 DBN is linearly proportional to the number of devices in the 1083 system. Hence, the computational requirement is increased as 1084 the system devices increase. Also, the proposed model has a 1085 low prediction accuracy in resource allocation.

1086
The authors of [116] have presented an unsupervised DL 1087 approach to compute an optimal transmit power for interfer-1088 ence management and sum-throughput maximization of an 1089 IoT system. The authors used a PCNet architecture to learn 1090 the features of training dataset and develop a DNN model for 1091 computing an approximate transmit power for a given channel 1092 realization. The authors reported an accuracy of 6.12% and 1093 5.92% for their model compared to the conventional opti-1094 mization algorithms. The model was deployed on the system 1095 devices by testing it in a deployable environment.  Disadvantages: The proposed PCNet model has a low pre-1100 diction accuracy for resource allocation with a small dataset. 1101

1102
The deep reinforcement learning approach is formed by com-1103 bining DNNs with reinforcement learning [121]. In a deep 1104 reinforcement learning (DRL) approach for resource man-1105 agement, an IoT application resource management problem 1106 may be mathematically modeled using the Markov deci-1107 sion process (MDP) framework. The MDP framework is 1108 employed to model the state space, the action space, and 1109 the reward of an agent. In this approach, a neural network 1110 is employed as an agent. The state space consists of the 1111 environment states, wheras the action space consists of a set 1112 of actions available to the agent in each environment state. 1113 At each discrete time with a step t, the agent interacts with 1114 the environment and observes the environment state from the 1115 state space and learns from its interaction with the environ-1116 ment. Then, the agent makes an action from the action set. 1117 Based on the action chosen, the agent receives either a reward 1118 or a penalty for making a good or a bad decision, respectively. 1119 Following this, the environment moves to a new state with a 1120 transition probability. The reason why the agent learns from 1121 its interactions with the environment is to compute an optimal 1122 policy that optimizes the overall accumulative rewards of 1123 different actions from the environment states. Examples of 1124 deep reinforcement learning approaches are deep Q-networks 1125 (DQNs), dueling DQNs, and deep Q-learning (DQL) [1]. 1126 Examples of the proposed deep reinforcement learning 1127 approaches for resource management are [122] and [123]. 1128 In [122], a dueling DQN model was presented to compute 1129 a transmit power solution for the secondary users (SUs) to 1130 enable them to accurately sense the spectrum usage in almost 1131    Table 5. Using DL approaches, the resource allocation 1178 decisions for obtaining solutions to resource management 1179 problems can be taken offline or their intensive online com-1180 putation could be minimized to reduce the use of power 1181 resources related to online computations as in the case of 1182 the optimization theory approaches. With DL approaches, 1183 optimal resource allocation solutions may be computed for 1184 IoT networks resource management problems with a low 1185 computational complexity.

1186
DL approaches are suitable for solving both convex and 1187 non-convex resource allocation problems in IoT networks and 1188 can provide resource allocation solutions in an almost real-1189 time manner.

1190
The prediction accuracy of the DL-based resource alloca-1191 tion approaches for IoT networks is still low and the level 1192 of prediction accuracy also depends on the quality of the 1193 available input data. Most DL-based resource allocation algo-1194 rithms for IoT networks have a large size and may not work 1195 well on most of the devices in IoT networks in practical 1196 applications due to their limited storage space.

1197
The supervised DL approach may be disadvantageous to 1198 obtain an optimal solution to some IoT applications resource 1199 management problems since its performance is technically 1200 bound by the resource allocation solution of the adapted 1201 conventional optimization algorithm.

1202
The unsupervised DL approach may be limited in perfor-1203 mance in terms of training and obtaining an optimal resource 1204 allocation solution, when applied to IoT applications resource 1205 management problems. The conventional loss functions used 1206 VOLUME 10, 2022 designed for classification and regression problems.  IoT networks. This section presents a review of different 1261 game theory approaches, their advantages, disadvantages, 1262 and different resource allocation solutions that are based on 1263 game theory.

1265
Game theory is one of the alternative approaches leveraged 1266 to solve resource management problems in IoT networks. 1267 Game theory is a strategic approach employed to model the 1268 behavior of devices as rational agents to optimize their gains. 1269 It can also be used to achieve a distributed resource allocation 1270 among a set of resource competitors, making it a powerful 1271 tool for solving resource management challenges in wireless 1272 IoT networks.

1273
Game theory approaches are applied mathematics that use 1274 computational approaches and optimization concepts to deal 1275 with decision making problems for the optimal control of 1276 resources by dynamically optimizing and adjusting a measure 1277 of performance [124]. It provides mathematical optimization 1278 frameworks that could be leveraged to manage scarce and 1279 critical resources in IoT networks, i.e. transmission time, 1280 bandwidth, and power resources.

1281
Generally, games are classified into two categories, namely 1282 cooperative games and non-cooperative games [125]. In a 1283 cooperative game, there exists a set of IoT devices that have 1284 agreed to work collectively with the aim of maximizing 1285 their overall objective function values. This type of game 1286 involves enforcing an agreement. To do this, a cooperative 1287 policy is used to introduce a binding agreement or a coalition 1288 among the devices, and this enables them to always cooperate 1289 to make decisions together and negotiating how to allocate 1290 resources, while no agreement exists between the devices in 1291 a non-cooperative game and they may consequently defect. 1292 Examples of game theory that falls under the cooperative 1293 games are the coalition games, repeated games, and bargain-1294 ing games [126], while examples of game models in the 1295 category of the non-cooperative game are the bid auction 1296 game theory, Stackelberg game theory, potential game theory, 1297 and the stochastic game [127].

1298
Both cooperative and non-cooperative game theory may be 1299 used for modeling as well as analyzing the resource allocation 1300 strategies developed for different heterogeneous IoT devices 1301 in a resource management problem. To compute optimal 1302 or near-optimal resource allocation solutions for resource 1303 allocation game problems, equilibrium solution concepts like 1304 Nash equilibrium (NE) and Stackelberg equilibrium (SE) are 1305 used for non-cooperative games, while the Nash bargaining 1306 solution (NBS) is used for cooperative games [127]. 1307

1308
In [128], a cooperative coalition game theory was employed 1309 to formulate a power control problem in D2D communica-1310 tion. In the study, the D2D users were modeled as players 1311 and a coalition game framework was developed to model the 1312 coalition of D2D pairs to form a group of D2D users and to 1313 encourage them to increase their objective function, which 1314 is sum rate. The D2D pair coalition is a mutual agreement 1315 between D2D users to share resource blocks (i.e., channels).   Table 6 to compare different game theory methods based 1384 on the addressed resource allocation problem, cost function, 1385 benefits, and disadvantages of the proposed optimization 1386 solutions.

1390
The optimization theory provides several mathematical pro-1391 gramming algorithms that could be employed to solve 1392 different categories of IoT networks resource management 1393 problems, for example convex and non-convex problems. But 1394 then, most of the resource allocation algorithms designed for 1395 resource management problems related to IoT networks using 1396 optimization theory are often faced with a high computational 1397 complexity related to computation power, computational time 1398 as well as storage space. This concern may increase the power 1399 consumption and the data transmission delay of the devices 1400 in IoT networks. This may eventually conflict with achieving 1401 the goals of time-critical IoT applications.

1402
The game theory provides mathematical optimization 1403 frameworks that could be leveraged to solve resource man-1404 agement challenges related to IoT networks to address the 1405 issues of transmission time, bandwidth, and power resources 1406 management. Also, it provides different equilibrium solu-1407 tion concepts to compute optimal or near-optimal resource 1408 allocation solutions for the resource management challenges 1409 in IoT networks. However, most of the resource allocation 1410 algorithms developed for IoT networks resource manage-1411 ment problems using game theory have a high computational 1412 complexity with a high computational power cost and long 1413 delays in real-time operations. Also, this concern may affect 1414 the performance of critical IoT applications in terms of data 1415 transmission delay, power efficiency, and throughput.

1416
The deep learning theory approach provides powerful 1417 mathematical tools that can be leveraged to obtain an opti-1418 mal or near-optimal resource allocation solution that are less 1419 costly. But then, most of the existing resource allocation 1420 algorithms based on DL approaches in literature are less 1421 efficient in terms of prediction accuracy. Some suffer from 1422 an increase in training complexity with a large number of 1423 VOLUME 10, 2022   The DL technique is promising to solve the resource man-1501 agement challenges arising in time-critical IoT applications, 1502 unfortunately, most of these solutions are inefficient due to 1503 computational complexity issues in terms of their computa-1504 tional storage space and computational power requirements 1505 when deployed on the constrained IoT devices. To enable DL 1506 resource management algorithms to be more efficient, future 1507 research is necessary to develop new methods for improving 1508 the computational complexity of DL resource management 1509 models. This line of research can benefit from the use of 1510 techniques that are suitable for improving the efficiency of 1511 DL models. Examples are knowledge distillation and pruning 1512 techniques [137]. This line of research is believed to signif-1513 icantly contribute to enabling DL-based resource allocation 1514 algorithms for the devices in IoT networks.   Even though the deep reinforcement learning approaches 1592 have promising potential to obtain optimal resource alloca-1593 tion solutions for IoT networks resource allocation problems, 1594 they are mostly confronted with a complexity issue during 1595 training. For example, as the number of IoT devices imple-1596 menting a deep reinforcement learning approach is increased, 1597 the training complexity of this approach may escalate. This 1598 concern may increase the computational resources required 1599 of the learning algorithm implementation devices. Also, 1600 it may hinder the goal of obtaining resource allocation solu-1601 tions in real-time operations as required by the time-critical 1602 IoT applications. To address the complexity issue associated 1603 with the deep reinforcement learning-based resource alloca-1604 tion approaches in IoT networks, future research is required 1605 to design and integrate efficient training techniques in such 1606 approaches to reduce the training complexity and computa-1607 tional resources. This line of research would contribute to 1608 efficiently managing the device power and speed. The DL-based resource management models require the def-1613 inition of hyperparameters like the number of hidden layers, 1614 the number of neurons in each hidden layer, the activation 1615 function(s), the optimizer, and the hidden layer parameters 1616 (e.g., weights and biases). The building of a good DL model 1617 for resource allocation prediction depends on the optimal 1618 tuning of the hyperparameters. To guarantee the realization 1619 of optimal hyperparameters to build a good model, future 1620 research need to consider the investigation and development 1621 of new optimization methods that can be used to determine 1622 optimal hyperparameters that enable the network to output a 1623 good solution. It will be interesting to also explore the use of 1624 different optimization techniques like Bayesian optimization 1625 and random search techniques. The use of multiple activation functions may be advantageous 1631 to build a DL-based resource management model with a good 1632 prediction accuracy for IoT networks. But then, there is a need 1633 to be able to select an appropriate activation function based on 1634 the system channel conditions. This requires the investigation 1635 and design of an optimal search strategy that can select the 1636 best activation function. A good idea could be to explore and 1637 exploit a tabu search method to design an efficient strategy. NVIDIA, and Xilinx [111], [113], [114]. The use of the 1658 SageMaker Neo tool has to do with using the tool to optimize

1670
This study has presented a comprehensive review of the 1671 use of deep learning approaches towards addressing the 1672 resource management challenges in IoT networks to improve 1673 the performance of IoT networks for various time-critical 1674 applications (e.g., industrial IoT, IoT-enabled water quality 1675 sensing networks, remote surgery). First, we collected the 1676 related published studies between 2012 and 2022 from the 1677 Scopus database. Subsequently, we conducted a bibliometric 1678 analysis of the collected studies to determine the current 1679 research focus in the field. Following this, we conducted a 1680 comprehensive review of the relevant studies to determine 1681 the existing research gaps. The bibliometric analysis and 1682 the comprehensive review revealed that research on the use 1683 deep learning approaches for solving resource management 1684 challenges in IoT networks is less common. Because of the 1685 usefulness of IoT networks in various applications and the 1686 resource limitations associated with the IoT networks as well 1687 as the need to efficiently use the limited available resource, 1688 the IoT networks require advanced and sophisticated resource 1689 management solutions to be investigated and developed to 1690 improve their data communication performance and opera-1691 tion lifetime. To fill this research gap, in this study, we intro-1692 duced the use of deep learning on account of its advantages 1693 over other artificial intelligence techniques (e.g., optimiza-1694 tion approaches and game theory approaches) in the context 1695 of computational complexity. Also, because of the lack of 1696 optimal solutions for most IoT networks resource manage-1697 ment formulations when using the conventional optimiza-1698 tion approaches, as such problems are mostly non-convex, 1699 this paper to compute resource allocation for the IoT devices in the IoT networks. Moreover, we discussed the fundamentals of deep learning approaches along with their uses, benefits, and challenges. Additionally, we point out important potential research directions and discusses the challenges  Electrical, Electronic and Computer Engineering, 2178 University of Pretoria. Her core pursuit is CMOS 2179 and BiCMOS mixed signal circuit design, mainly 2180 in readout and signal processing for imaging arrays, biologically inspired 2181 electronic circuit applications, and specialized devices for analog IC design. 2182 Her most recent endeavor is printed electronics sensors and circuits, par-2183 ticularly for water quality monitoring, but also in point-of-use diagnostics. 2184 She is an inventor on three patents, has published 22 journal articles, and 2185 presented and coauthored 65 peer-reviewed conference proceedings papers. 2186 She has lectured short courses to industry in her field of specialization, 2187 worked on 31 projects for the South African industry, writing 131 technical 2188 reports in the process, and contributing to 32 integrated circuit chips. The 2189 contributions of her work to the South African innovation system spans 2190 university teaching, applied research, and leading industrial development of 2191 prototypes and products. She is registered as a Professional Engineer with 2192 the Engineering Council of South Africa (ECSA). She has been a member 2193 of both the technical and the organizing committee for several international 2194 conference events. 2195 2196 VOLUME 10, 2022