Self-Competence Learning for Bandwidth Slicing and Confidential Computing Requirement

This work proposes a Self-Competence Learning bandwidth slicing method (SCL) to determine the quantity and position of each slice’s spectrum resources considering the channel diversity and varying bandwidth requirements. SCL obtains high throughput in the following aspects: (1) SCL provides artificial intelligence (AI) based functionality to determine the quantity and position of the spectrum resource, (2) SCL can be trained solely by self-competence without any assistance or labeling by manual operations, and (3) SCL provides confidentiality without compromising on training performance in collaborative learning environment. Simulation results demonstrate SCL improves the 10% overall throughput and saving up 25% bandwidth resource for AI model merging.


I. INTRODUCTION
Network bandwidth slicing enables the 5G/B5G service providers to accommodate heterogeneous use cases over a common underlying physical telecommunication infrastructure, such as the enhanced mobile broadband (eMBB), ultra-reliable and low latency communications (URLLC), and massive machine type communications (mMTC) [1]. In order to maximize the spectrum efficiency (SE) and guarantee superior quality of experience (QoE), the service providers have to intelligently allocate sufficient spectrum to each slice according to the channel diversity and varying bandwidth requirement of the service types in different slices. This is called inter-slicing resource management problem [2], which is a maximum-matching problem based on bipartite graph theory to allocate slices to channels [3][4], of which the problem statement: Given a bipartite graph = ( , ; , ) with I slice vertices (S), J channel vertices (C), and each edge ∈ has a nonnegative modulation coding rate , ∈ MCS where ∈ and ∈ . The target is to find the matching ∈ that assigns each channel j to slice in order to maximize the value function, e.g., overall throughputth( ), under the constrain that the allocated bandwidth to each slice i will not exceed its requirements ∈ .
Various works provide valuable knowledge on building bandwidth slices, however, there exist two issues. First, the conventional spectrum resource allocation method consists of two phases: first determine the quantity of bandwidth resource each slice would get, and then determine the position of the spectrum. Qi et al [2] apply deep reinforcement learning to decide the quantity of bandwidth to each slice according to the network status. Li et al [5] consider the user mobility and traffic patterns and adopt long/short-term memory to predict the possible demanding quantity of bandwidth of each slice. Hua et al [6] use a generative adversarial network to stabilize artificial intelligence (AI) training procedures. These works reach a great achievement, however, they merely determine the quantity of spectrum without indicating the position of the spectrum.
Second, the AI-based method should cooperate existed or traditional methods, instead of simply replacing them. That is, besides the general models [2][3][4][5][6] to maximize the throughput, there are also economic model [7] to maximize the profit of service provider, robustness model [8] to achieve high availability of telecommunication network, and confidential computing model [9] to provide confidentiality of private data without compromising on training performance. Replacing the existed methodology embedded in the current system may be costly and impractical, and the proposed AI-based method should be capable of cooperative learning and consider the privacy issues of confidential computing requirements.
This study proposed a Self-Competence Learning bandwidth slicing method (SCL) that applies one-step functionality, instead of two-phase allocation, to determine the quantity and position of spectrum resources to each slice considering the channel diversity and varying bandwidth requirements. SCL differs from previous works [2][3][4][5][6][7][8] in several aspects. First, SCL is the first work, to our best knowledge, that applies AI-based functionality to simultaneously determine the quantity and position of the spectrum resource. Second, SCL is trained solely by selfcompetence reinforcement learning without any assistance by manual operations: building multiple AI agents of a single neural network structure, starting from random strategy making, competing, and learning from each other. Third, SCL enables service providers to collaboratively train a shared model without knowing each other's local data in a confidential computing environment. The design of SCL and the performance evaluation are provided as follows.

II. THE SCL DESIGN
SCL applies one-step allocation to simultaneously determine the quantity and position of spectrum resources to each slice considering the channel diversity and varying bandwidth requirements. The one-step allocation of SCL adopts an end-to-end function design to transform the INPUT directly to the OUTPUT by AI-based methodology. On the other hand, SCL adopts a self-competence learning procedure by creating multiple AIs to compete and learn each other to maximize the overall throughput without any assistance by manual operations. Self-competence learning is also compatible with collaboratively training without knowing each other's local data in a confidential computing environment. The detailed design regarding the End-to-End Function Design and the Self-Competence Training Procedure is provided in the following subsection.

A. ABBREVIATIONS AND ACRONYMS
The one-step allocation of SCL adopts end-to-end function consisting of four basic components including INPUT, Forward Function (FF), Reward Value (RV), and OUTPUT. Fig.1 shows three steps of SCL to transform the INPUT to the OUTPUT.
First step is that, as shown on the left side of Fig.1, according to the inter-slicing resource management problem statement described in the introduction section, we have the INPUT including the network status MCS and the bandwidth requirements : Second, as shown in the middle of Fig.1, SLR applies Forward Function (FF) to transfer the INPUT, i.e., and R, to the Reward Value (RV). The Reward Value , ∈ is the expected throughput of each possible slice-channel matching denoted by where vi,j is the expected throughput while assigning j-th channel to i-th slice. The FF consists of full-connected, convolution, and pooling operations. As shown in Fig.1, the input/output provides a point of input/output with an identity activation function, e.g., min, max, category index. The kernel applies a parameterized weight to the input or output. The hidden cell is a kernel that attenuates the input and forms a foundation to generate the output. The convolution or pooling cell is to aggregate and extract the features from the inputs. The parameters in the FF will be adjusted by the loss function, gradient operation, and corresponding selfcompetence strategies described in the next subsection. SCL then applies argmax() operation to find the matching ∈ in order to maximize the value function, e.g., overall throughput th( ) . That is, SLR explores the index mj owning the highest reward value along each channel j by = argmax ([ 1, , … , , , … , , ] ) (1).
The means channel j is assigned to slice with highest expected throughput located on channel j. Then SLR finds all channel-slice matching ∈ denoted by Since each slice owns the highest expected throughput for channel j, the SLR infers the matching ∈ provides maximized throughput and treats it as the OUTPUT.
Here using a simple example to demonstrate how SCL transforms the INPUT, i.e., and R, to RV. At first, SCL flattens and combines and R into a one-dimension input array = [ 1,1 , … , , , 1 , … , ] . Then, given the FF using only one full-connected operation, SCL applies fullyconnected operation aggregates information from that matters the most and then generates RV by (2): ] is the bias vector.
The Equation (2) first executes the dot operation ( • + ) to acquire a one-dimension array, and then applies Reshape function to reshape the one-dimension array to twodimension × array. In addition, and will be iteratively adjusted in the training procedure to ensure the channel-slice matching ∈ can achieve the highest throughput.
The design of Forward Function can be adjusted to balance the computation complexity and performance. The more complex the operation design, the higher throughput it achieves. However, the training time and computation time are getting higher for a deeper network.
In addition, the FF design of SCL can cooperate with the previous works [2][3][4][5][6][7][8] by setting the bandwidth requirements to the quantity scheduled for each slide.

B. Self-Competence Learning Design
The self-competence learning strategy of SCL is derived from the natural selection process. That is, as shown in Fig.2, SCL adopts three phases to ensure multiple AIs compete/learn each other and then iteratively evolve from current generation of models into a new generation of the best model which can provide maximizes throughput.

FIGURE 2. SCL Self-Competence Training Procedure
For the first phase, SCL randomly generates multiple individual Forward Functions ∈ , and then uses the same samples to compete and learn from each other: Step 1: For the competing, each computes its own RV k and M k according the same input R and MCS. VOLUME XX, 2017 1

FIGURE 3. Generate a second generation population of SCL
Step 2: SCL then pick up the with the highest throughput th( ).
For the second phase, after sufficient competing and learning iterations of the first phase, SCL applies the elitism selection to allow the best FF from the current generation to carry over to the next generation, and to eliminate the remaining FF.
The third phase generates new FF and replaces the eliminated FF by mutation, cloning, and meiosis of currently chosen elites FF, as shown in Fig. 3. This phase attempt to change the heritable traits of the current FF population over generations to get better parameters and allocation strategies with partial reproduction are used.
For changing the heritable traits, SCL applies mutation to maintain genetic diversity from one generation of a population of chromosomes, i.e., the hidden kernel of SCL, to the next. As shown in the left part of Fig. 3, the mutation of SLR uses Gaussian distributed random value to the weight of the randomly chosen hidden kernel. For the partial reproduction operation, SCL combines the genetic information of two elite parents (FFs) to generate new offspring FF. The combination process is similar to meiosis which is a sexually-reproducing organism used to produce the gametes. As shown in the middle part of Fig. 3, SCL randomly chooses one hidden kernel from the elite FFs and builds a new generation of FF.
After the third phase of generating a new generation, SCL then goes to the first phase and then iteratively evolves into a new generation of the best FF.
In addition, the meiosis design of SLC is similar to and is capable to cooperate with the federal learning method such as AvgFed [9] methodology. Furthermore, the meiosis design can also use partial parameters, instead of all parameters, from other FF to save the bandwidth for model sharing and merging in a confidential computing environment.

III. PERFORMANCE EVALUATION
Simulations were performed to compare the proposed SCL algorithm with non-AI based method, i.e., Hungarian Algorithm based method (HA) [3,11], in terms of total throughput, since there is no AI-based method to determine the quantity and position of the spectrum simultaneously. We also evaluate the performance of SCL in a confidential computing environment especially for different parameters sharing strategies in order to save the bandwidth of model merging.
The simulator was extended from the ns2 VANET and OFDMA simulator [12][13]. The simulation environment was a 10MHz LTE system with three types of services (i.e., eMBB, URLLC, mMTC), with 300 UEs and three base stations (BSs) capable of sharing models in a confidential computing environment. The first BS applies Hungarian Algorithm to allocate bandwidth resources to each slice. The second one adopts SCL with 8 FFs for self-competence learning, 4 FFs owning lower throughput will be eliminated and replaced per generation. The third BS is equipped SCL with only one FF to receive a model from the second BS for performance evaluation of SCL confidential computing. This means the third BS merely builds its own FF by acquiring the parameters of FF from the second BS without knowing the samples trained by the second BS. According to the received SNR, various MCSs were used, including QAM1/2, 16QAM1/2, 16QAM3/4, 256QAM1/2, 256QAM3/4.

A. Performance Comparison between HA and SCL
To investigate the effect of average requested bandwidth of each slice on total throughput, the aggregated data arrival rate of each slice was varied from 4 Mbps to 64 Mbps, and the number of slices are 10 in this scenario.   Fig. 4(a) presents the total throughput achieved using HA and SCL. SCL outperforms HA with about 9.3% improvement when the requested bandwidth increases for two main reasons. First, HA does not consider the requested bandwidth from each slice, and then some slice with the best channel quality will acquire the largest spectrum resources, even this slice does not have so much requested bandwidth and leave these oversupplying resources to be unused. Second, SCL, on the other hand, uses the channel quality and requested bandwidth from each slice as input, and applies self-competence methodology to ensure the AI iteratively evolves into the model that can provide the highest overall throughput.
The effects of the number of slices on the total throughput were also investigated. The number of connections was varied from 3 to 10 with the same overall data arrival rates, i.e., 280 Mbps. Fig. 4(b) reveals that SCL outperforms HA with about 7.3% to 9.3% improvement along with the increasing number of slices. Under the same overall data rate, the larger the number of slices, the smaller the bandwidth requested by each slice and the fewer the spectrum resource area of each slice. A slice with fewer spectrum resources provides SCL with a higher probability to place the slice on high-quality channels.

A. Performance Evaluation for Confidential Computing
To evaluate the meiosis design of SLC is capable to cooperate with the federal learning such as AvgFed [9] methodology and, furthermore, and can save bandwidth for model sharing/merging in confidential computing environment. The simulation environment settings are 10 slices with overall 280 Mbps aggregated data arrival rate. Fig 5 demonstrates that using AvgFed methodology needs 8 model exchange times to achieve 234 Mbps overall throughput, which is close to the performance of second BS and ensure that the SCL model is capable of model sharing for confidential computing environment.
We furthermore conduct an experiment that the second BS randomly selects partial parameters (from 25% to 75% parameters of the second BS's FF) and shares them with the third BS. This experiment attempts to observe that SCL only required partial parameters of sharing to achieve the similar performance of complete parameter sharing. Fig 5 reveals that using only 75% parameters (AvgFed 75%) can achieve 231 Mbps after 8 exchange times and 234 Mbps at 9th exchange time. This means SCL meiosis design of SLC can achieve similar performance of full model sharing of FedAvg by saving 25% bandwidth of model sharing/merging procedure.

IV. CONCLUSION
This study introduces a Self-Competence Learning bandwidth slicing method (SCL) that applies one-step functionality, instead of two-phase allocation, to determine the quantity and position of spectrum resources to each slice considering the channel diversity and varying bandwidth requirements. The simulation results reveal that SCL provides 10% improvement of overall throughput than the previous works. SCL also achieves similar performance by saving up to 25% bandwidth resource for model sharing and merging procedures in a confidential computing environment.