Generalizable Machine-Learning-Based Modeling of Radiowave Propagation in Stadiums

Providing high throughput and quality of service in modern stadiums necessitates the placement of hundreds of access points (APs). Optimizing the locations of APs in such venues via measurements requires significant resources. Even simulation methods, such as ray-tracing, can be computationally costly. We provide a solution to this problem by building a propagation model based on machine learning (ML) that rapidly predicts received signal strengths in stadiums. We train the model with a small set of simulated data generated by a ray-tracer. We use input features, such as the electrical distance between the transmitter and the receiver, and the antenna gain along the direct path between the two, to generalize to new transmitter locations, antenna patterns and stadium geometries. Geometry and pattern generalization have not been included in existing propagation models for stadiums. Finally, we present a novel sampling approach for the input features in a given stadium, ensuring the computational efficiency and accuracy of the ML model. The results demonstrate the accuracy of our propagation model for new transmitter locations, patterns and stadiums. The trained model is also considerably faster than a ray-tracer, making it an efficient tool for resource planning tasks, such as optimal placement of APs.


I. INTRODUCTION
L ARGE sport arenas and stadiums can accommodate tens of thousands of spectators.Big sport events and specific instances within an event can lead to abrupt surges in the usage of the existing communication infrastructure.To offer high quality of service (QoS) to such extreme, but not uncommon, occurrences, efficient planning based on accurate modeling of the communication channel is important.
Effective propagation modeling requires the accurate prediction of the received signal strength (RSS) at a given location.This can be accomplished by extensive measurement campaigns.However, modern communication systems operating in stadiums utilize hundreds of transmitting antennas.In such cases, on-site measurements can be expensive and time-consuming.Alternatives include empirical propagation models or computational methods, such as ray-tracing (RT).However, empirical propagation models still require measured data to calibrate parameters such as the path loss exponent [1].Moreover, RT simulations can be computationally demanding for complex stadium geometries, with hundreds of antennas to simulate at several frequencies.
In the literature, all these aforementioned approaches have been employed for propagation modeling in stadiums.Measurement campaigns were undertaken in [2], [3], [4], [5], [6], [7], [8], [9], [10], RT simulations were used in [11], [12], [13], [14], and propagation models were derived in [4], [8], [15].Moreover, the geometry of the stadiums also varied, from small, open-track soccer stadiums [3] and closed basketball courts [9], to larger venues, such as multi-level stadiums, with capacities of tens of thousands of spectators [6], [7], [10].However, the role of parameters such as the transmitter (Tx) configuration (position, orientation and pattern) and the geometry of the stadium have not been included in existing models.Few papers derived propagation models to predict RSS at different locations within the same stadium, under a pre-defined Tx configuration [4], [8], [10], [15].However, resource planning tasks, such as optimal placement of APs, require models that can generate fast and accurate predictions under multiple Tx configurations.Also, this should be accomplished efficiently and accurately in a variety of stadium geometries.
Machine learning (ML) techniques for propagation modeling have been developed, and extensive work is available in the literature [16], [17].Similar to empirical propagation models, ML-based propagation models are derived by fitting a set of input features x to their corresponding observations (targets) y.This is done iteratively, by learning parameters (weights) w such that y = f (x, w).This process is called training.Thus, a trained ML model consists of a set of weights that, given an input sample i and features x i , can generate an output ŷi to approximate the actual y i .A desirable feature of a trained ML model is generalizability, i.e., accuracy when applied to cases outside the training set [18].In the context of propagation in stadiums, this includes new transmitter-receiver (Rx) locations, antenna patterns, frequencies of operation and stadium geometries.Geometry generalizability has been recently investigated for indoor environments [19], [20], but not for propagation in stadiums.
In this paper, we build a propagation model for stadiums employing ML techniques.Our ML model, an artificial neural network (ANN), is trained with data generated by a shooting-and-bouncing-rays (SBR) ray-tracer [21].By selecting useful input features, the model can be utilized in a variety of different propagation scenarios, involving different Tx and Rx positions, antenna patterns and geometries.To the best of our knowledge, this is the first propagation model that generalizes to both new antenna patterns and geometries in stadiums.We also introduce a novel and systematic method for optimally sampling the training features of the ML model.Thus, we limit the number of RT simulations required for training, but also ensure the generalization accuracy of the model.Although the proposed input features and sampling method are employed for propagation in stadiums, they are transferable to other geometries, such as indoor environments.The proposed (trained) model can rapidly estimate the RSS levels in a stadium.Hence, it can dramatically accelerate the performance of tasks, such as coverage and capacity estimation, optimal placement of APs, and interference management, for a variety of different Tx configurations and geometries.
The structure of the paper is as follows.In Section II, we provide a brief literature overview of propagation modeling in stadiums.We focus especially on how the pattern, position and orientation of antennas can influence RSS levels in a stadium.In Section III, we introduce the workflow of our ANN-based propagation model.We describe the data generation process, the architecture of the ANN, as well as its input features.In Section IV, we explain how to sample those input features to create a generalizable and computationally efficient propagation model.In Section V, we employ the proposed sampling process to select the training parameters of the ANN, such as the positions and patterns of the transmitting antennas.Section VI presents generalization results in a fixed stadium geometry, where our trained model is evaluated on different Tx configurations.Finally, Section VII demonstrates the generalizability of our model to new stadium geometries.
A communication system employed in a stadium must provide high QoS to a large number of users.The number of APs, their locations, the types of antennas used and their orientations, are important for designing an effective system that meets this goal.Moreover, beamforming and sectorization are widely used in stadiums.In the following, we will further discuss these design options and offer insights on popular practices presented in the literature.

B. AP PLACEMENT
Most APs are placed above the seating area.Over-the-seats APs allow for line-of-sight (LoS) propagation between the antenna and a user in the stands.Moreover, such a placement mitigates human blockage losses [6], [8].Regarding the orientation of the transmitting antennas, the main approach is to direct the main beam of the antenna to the center of its sector [11], [15].

C. ANTENNA PATTERNS
Most existing work focuses on specific antenna patterns.Useful comparisons between narrow-and broad-beamwidth antennas were made in [11], [12], [13], [15].It was found that the directive (narrow-beamwidth) antennas offered better signal-to-noise and interference levels at high-density AP deployments (124 and 344 APs), compared to broaderbeamwidth antennas.On the contrary, at low-density AP deployments (62 APs), the broader-beamwidth antennas performed better thanks to their larger coverage footprint.Even though these comparisons offer guidelines on what antenna to select for specific coverage profiles or AP density deployments, they are only valid for the specific antennas and geometries included in these studies.

D. SECTORIZATION
A variety of different sectorizations has been used in the literature for propagation in stadiums [14].In elevation sectorization (also called level or concourse sectorization), different sectors are formed on the elevation plane of the stadium and mapped to different stadium levels [8], [10], [11].
In the azimuthal sectorization, the stadium is divided into sectors on the azimuthal plane [10], [15].Then, the transmitting antenna points towards the line connecting the center of the stadium to the location of the sector [11], [13].A hybrid sectorization scheme can also be used, utilizing both elevation and azimuthal sectorization [11], [12], [13], [15].

E. CROWD PRESENCE
Crowd presence can influence the RSS levels in a stadium [7], [8], [12].For example, self-body blockage (blockage by the person holding the terminal) can diminish user capacity, especially if the AP is located behind the user [12].The same holds for blockage from nearby spectators, especially when the person holding the user terminal is seated while other users are standing.Quantifiable results regarding human blockage, such as those reported in [7], can be incorporated into the fade margin of the communication system, ensuring reliability and QoS standards.

F. PATH LOSS MODELING
A couple of empirical path loss models were also derived by fitting measured data.As an example, the authors in [4] used the log-distance path loss model: where PL(R) is the path loss computed at a distance R from the transmitter, PL(R 0 ) is the path loss computed at a reference distance R 0 , n is the path loss exponent, and X is a random variable that accounts for the fading characteristics of the channel.Since stadiums are LoS-dominant environments, the free space path loss (FSPL) formula without shadow fading, i.e., setting X = 0 and n = 2 in (1), has also been employed [15].A variation of (1) was used in [8], where the authors computed the received power at distance R as: where P T is the transmitted power, and P R (R) the received power at distance R. The gains G T and G R are the antenna gains of the transmitter and receiver respectively, L add is a combination of the polarization, absorption and antenna alignment losses, L nLoS are losses for non-LoS (nLoS) cases that correspond to the measured human blockage losses, and f is the frequency of operation.Finally, in [10], the authors used variography and Kriging to predict RSS at unmeasured locations.Variography was used to extract the variogram of the measured path loss, by modeling its spatial dependency.Then, Kriging used the variogram to compute the RSS at new locations in the stadium.Even though the accuracy of the model was high, it was only evaluated at different Rx points, assuming a fixed Tx configuration and stadium geometry.None of the described models was evaluated under diverse propagation scenarios, including new transmitting locations, antenna patterns, or in new stadiums.

III. ML-BASED PROPAGATION MODEL
In this section, we explain the building blocks of the MLbased propagation model.We start by discussing the data generation process that involves the ray-tracer.We then present the input features used, and explain how they are computed.Finally, we discuss the workflow and architecture of the ANN.In Table 1, we present the main notation used in the current and following sections, along with a brief description for each symbol.

A. DATA GENERATION
Our ML model can be trained with measured, synthetic or a combination of measured and synthetic data.In this work, the model is developed from synthetic data produced via a ray-tracer.Despite a large number of measurement campaigns on propagation in stadiums [2], [3], [4], [5], [6], [7], [8], [9], [10], no actual data are directly available.However, the main objective of our work is to develop an accurate, efficient and generalizable model.The assessment of our model with respect to these three properties can be readily made on a fully synthetic dataset, by extracting mean average errors (MAEs) for varying antenna configurations The ray-tracer requires a facetized model of the environment, the locations of Tx and Rx points within the stadium, the frequency of operation, as well as other RTrelated parameters, such as the number of ray interactions allowed per ray.We denote the input to the RT solver with the vector x RT .The RSS values at the specified Rx points are computed by RT, and converted into path gain (PG) values.These are used as target values to train the ANN.Unlike the received power P R at distance R from the transmitter, PG does not depend on the transmitted power P T .Having PG, P R can be easily computed from: The data generation process is invoked twice: first, to generate the training data and second, to generate the test data.Test data samples are not used during the training of the ANN.They are used to evaluate the generalizability of the trained model.In our case, the test dataset consists of Tx and Rx locations, frequencies, antenna patterns and stadium geometries, that are not included in the training set.To differentiate between the training and test datasets, the input and output of the ray-tracer during training is denoted with x RT,tr and y tr respectively, and during testing with x RT,ts and y ts .The process is visualized in Fig. 1.

B. INPUT FEATURES
For each Rx point, the following 4 input features are computed (see Fig. 2): • The electrical distance d el = d λ between Tx and Rx, where d is the physical distance (in m) between Tx and Rx, and λ is the wavelength (in m) for a given frequency.This feature jointly captures the influence of the Tx-Rx distance and the operating frequency on the RSS.Thus, it helps the model generalize with respect to different frequencies, and Tx and Rx placements.
• The θ and φ angles (spherical coordinates) of Rx with respect to the Tx coordinate system (see Fig. 2).These features help the model generalize with respect to different Tx-Rx orientations, as well as new antenna patterns.
They also help the model distinguish symmetric and non-symmetric patterns.• The gain G(θ, φ) at those angles.This feature helps the model generalize with respect to new antenna patterns.The θ and φ features are computed based on the coordinates of the Tx and Rx antennas, as well as their orientation.Note that the computation for the G(θ, φ) feature involves only the direct ray from the transmitter to the receiver.Note also that the network does not learn absolute values of φ and θ with respect to a global coordinate system; all angle values are computed relative to the orientation of the transmitting antenna.Hence, the features are suitable for generalization to other geometries and Tx configurations.
All features are normalized to have a zero mean (μ = 0) and a standard deviation of 1 (σ = 1).The normalization is applied to each input feature x i , i = 1, 2, 3, 4 according to: where μ i and σ i are computed for the corresponding i-th feature among all samples of the training set.Normalization ensures that all features have comparable ranges.This decreases the bias of the network with respect to specific features, and speeds up the training process.Note also that the orientation of the antenna is not explicitly given as an input to the model, but is captured by the φ, θ and G(θ, φ) features.Hence, a rotated antenna pattern is treated as a distinct pattern by the network.

C. ML MODEL WORKFLOW AND ARCHITECTURE
The model is trained by data in the form of (x, y) pairs, where x is a vector containing the input features for a given Rx point in the stadium, and y is the scalar PG value converted from the RT-computed RSS at that point.For a single transmitter, the total input to the ANN is a matrix X tr ∈ R N tr ×N fts for the training set, and X ts ∈ R N ts ×N fts for the test set, where N tr and N ts is the number of input samples (Rx points in the stadium) for the training and test cases respectively.Also, N fts is the number of input features per sample (N fts = 4).Likewise, the total target y for the ANN comprises all scalar PG values and is denoted by the (column) vector y tr ∈ R N tr ×1 for the training set, and y ts ∈ R N ts ×1 for the test set.The target values y are generated by the ray-tracer for both training and test data.The corresponding x vectors are pre-computed.The input features in x are not the same as the input parameters used by the solver, x RT , hence the difference in notation.For example, the spatial coordinates for a Tx antenna must be specified in the ray-tracer.However, the ML model does not use this information, since information that depends on a specific coordinate system will not lead to a general model.The workflow of the ML model can be seen in Fig. 3.
The ML model we use is an ANN [22].This is a simple ML network, but sufficient in a case where the number of input features is small.The ANN consists of 2 hidden layers of 600 and 300 neurons, respectively, followed by a single output node corresponding to the PG for a given input sample.The architecture of the ANN is visualized in Fig. 4. The ANN is trained iteratively over time by backpropagating the errors computed at its output, and by appropriately adjusting its weights in order to minimize its cost function [22].The cost function is the mean squared error (MSE) between its output ŷ and the simulated target values y.Then, the trained ANN can generate predictions for new scenarios (new Tx-Rx configurations, frequencies, etc.).This is done almost instantaneously, without the need to run any RT simulations.In the testing phase, we evaluate the ANN by computing the MAE between its predictions and the corresponding target values.The workflow of the training and test process of the ANN, along with the definitions of the error metrics, can be seen in Fig. 5.

IV. INPUT FEATURE SAMPLING
Having explained the input features, we now discuss how to sample them.Note that the ML model has to strike a balance between generalizability and computational efficiency.Training the ANN with insufficient data compromises its accuracy on the test data.On the contrary, utilizing excessive  amounts of data will severely impact the data generation and training time of the model.To discuss these topics, we introduce a stadium geometry used to train our model in Fig. 6.This stadium, hereafter referred to as Stadium 1, is open-air and single-level.The geometry corresponds to an actual stadium and is 183 m long, 144 m wide, and 10 m high.We start this section by presenting the main objectives and the outline of the sampling process.We then describe in more detail how each input feature is sampled.Finally, we formulate the sampling process as two algorithms.

A. OBJECTIVES AND OUTLINE OF THE SAMPLING PROCESS
The goal of the sampling process is to provide training samples that are representative of the testing cases expected to be processed by the trained ANN.This is accomplished by ensuring that the range of the sampled training input features covers the range of possible test features.Moreover, the entire range should be adequately sampled, leaving no sampling gaps.This translates to covering the whole range of [0 • , 180 • ] for the θ feature, and the [0 • , 360 • ) for the φ feature.Regarding the d el and G(θ, φ) features, their range can be chosen more freely, based on the desired goal.
The outline of the sampling process is as follows.We initially place N R Rx points close to the stands of the stadium.Based on these points, we select the necessary number of transmitter locations and orientations (Tx-LORs) to adequately sample the θ and φ features.If our sampling requirement is not met, we increase the number of Rx points and run the process again, re-selecting Tx-LORs.A similar process is followed to select antenna patterns for each chosen Tx-LOR.The Tx-LORs and antenna patterns chosen by this process are used to train the ANN.Finally, we adjust the frequencies assigned to each selected training transmitter, to adequately sample the d el feature.
Given a selected set of receivers and transmitters (the selection process will be discussed next), we ensure adequate feature sampling as follows.We first compute the values for a given feature and quantize them into N b bins, thereby generating a feature distribution of the values.We then compute the average number of sampled values per bin Npts,b as: We finally require that every bin i b has enough sampled points according to: Plotting the histogram of the generated distributions offers a quick way to validate the sampling requirement.Two such examples are presented in Fig. 7, for the d el and φ distributions generated by the Rx and Tx selection used to train our ANN.Even though these distributions are not uniform, they both satisfy (6) within their respective range of values, leaving no sampling gaps.For a given Rx selection, different combinations of Tx-LORs can achieve the sampling criterion of (6).The optimal selection is based upon comparing the generated feature distribution of each of the combinations with the respective test feature distribution.The latter is not known during the training of the model.However, due to the physical constraints on Tx placement in a stadium, we can anticipate possible test locations.Similar constraints apply  to the positions of the users, being close to the stands.Hence, a set A of candidate receivers and Tx-LORs is created.A receiver in A is parameterized by its location coordinates (x r , y r , z r ), while a Tx-LOR is parameterized by the transmitter coordinates (x t , y t , z t ), and its orientation as denoted by the corresponding horizontal (φ tilt ) and vertical tilts (θ tilt ) with respect to the global coordinate system (Fig. 8).Set A comprises our validation set, since we use it to validate our design choices during the training of our model.It also acts as a proxy for the test set.Hence, by optimizing the network to be accurate on the validation set, we expect it to be accurate for similar test scenarios.
Having generated the validation distribution, we need to compare it with the training distribution for each feature.This is done by computing (per feature) the Z score as [23]: In (7), xtr and xval are the means of the training and validation feature distributions, whereas σ 2 tr and σ 2 val are the variances of the 2 distributions, and N tr and N val is the number of samples in each one of them.The Z score measures the distance of the distributions in the feature space.A similar process is followed to select the training antenna patterns to be assigned to the training Tx-LORs.
Iteratively optimizing for the Z metric leads to an optimal choice of training Tx-LORs and patterns.These parameters generate training distributions, such as those presented in Fig. 7, that are close to the validation distributions.Note that the spatial distribution of Rx and Tx points in the stadium influences the generated distribution of both d el and the angle (θ and φ) features.The orientation of a transmitter also influences the generated θ and φ features.Finally, the generated G(θ, φ) values depend on the selected antenna patterns.In the following, we explain in greater detail how each input feature is sampled.

B. RECEIVER PLACEMENT
As already discussed, set A includes a set of Rx and Tx locations.Instead of jointly optimizing the positions of both Tx and Rx points, we assume a specific spatial distribution for the Rx points.We then optimize the position of transmitters for the given Rx distribution.This has two benefits.First, it allows for a realistic placement of Rx points close to the stands of the stadium.Second, it is faster to optimize the position of transmitters, which are significantly fewer than the receivers.
Regarding the Rx placement process, the following steps are used to place a receiving point R(x r , y r , z r ) in the stadium: • First, an inner and an outer rectangle that enclose the stands are drawn (see Fig. 6).Then, the distance of each of the inner and outer rectangles with respect to the center point of the stadium O(x o , y o , 0) along the x and y dimension is computed.
, where x 1 , x 2 , y 1 , y 2 are defined in Fig. 6.Its z r coordinate is given as z r = z o + z n , where the z o coordinate is computed based on the slope of the stands at point R, and z n is a variable following the normal distribution with a mean of 1 m and a standard deviation of 0.25 m, i.e., z n ∼ N(1, 0.25 2 ).It is added to the height z o of each receiver to account for the variability of a receiver's height, and further generalize the model.A top-down view of 1500 randomly generated Rx points is shown in Fig. 9.

C. TRANSMITTER PLACEMENT -ANGLE FEATURES
Regarding the placement of transmitters, a similar process is followed, where transmitters are uniformly placed around the stadium, within the rectangular frame defined in Fig. 6.They are placed at variable heights, both above and behind the seating area, as well as within it.The transmitters are also assigned variable orientations, so that φ tilt ∼ U [0 • ,360 • ) and θ tilt ∼ U [90 • ,130 • ] .Note that the placement of a transmitter need not be realistic.A transmitter can be placed at the center of the stadium if that helps the model learn better, even if the transmitter cannot be physically placed there.
We now need to identify the training Tx-LORs in the stadium.Possible candidates are selected from set A, based on the θ and φ feature distributions they generate.Given the specified Rx points and the Tx-LORs in A, the mean φi angle for the i-th Tx-LOR can be computed as: where we average the φ values over the N R Rx points placed in the stadium.Then, the mean φT angle over all N T Tx-LORs in A is: To select the training Tx-LORs, we compute the Z score for the i-th Tx-LOR in A as in (7): The same process is followed for computing θi , θT and Z θ,i scores.Then, for each angle feature we select 3 Tx-LORs.
The first is the one with the smallest absolute Z score, corresponding to the best Z score achieved.The remaining 2 are those with the worst Z scores; i.e., the largest and lowest Z scores achieved in (10).The reason for selecting training Tx-LORs with poor Z scores is that they help the network capture outliers in the feature space.This provides more diversified cases in total, compared to only using the Tx-LORs with the best Z scores.Next, we check if our Tx selection satisfies the sampling criterion of (6).If not, we double the number of Tx-LORs selecting the 2 best and 4 worst Z scores.If that selection does not satisfy our sampling requirements, we do not increase the number of Tx-LORs further.Instead, we increase the number of Rx points placed in the stadium, and re-select Tx-LORs for the new Rx points.This is done for computational reasons, since selecting a large number of training Tx locations will require an equally large number of RT simulations.In general, we select up to 3k (k = 1, 2) Tx-LORs for each angle feature, for a total of 6 or 12.The transmitters selected by this process comprise set A * .

D. GAIN FEATURE
The (θ, φ) angles should lead to meaningful gain values for the network to learn from.Moreover, the training and test gain ranges should also be comparable.For that reason, we normalize all antenna patterns in the −45 dBi to 0 dBi range.Since we do not know the possible test gain range a priori, we make sure that the training gain distribution adequately covers the selected normalized gain range.
As discussed in Section II, most transmitting antennas in stadiums are directive.Hence, the aim of the network is to generalize to different directive patterns, while still being able to capture omni-directional antenna patterns.For that reason, we select a pool of 12 antenna patterns of various directivities and shapes, comprising our pattern set P. These antenna patterns correspond to antennas that have been actually deployed in stadiums.We expand P with rotated versions of the initial patterns, both in the azimuth and elevation planes.We then select training patterns following a selection process similar to what we followed for the angle (θ, φ) features.We successively assign each pattern in P to all the Tx-LORs in A * found before, and compute the mean gain Ḡp for the p-th pattern as: where we average all G p,i * ,j * = G p (θ i * ,j * , φ i * ,j * ) values over the number of N R * receiving and N T * transmitting locations we selected before.Then, similar to (9), we compute the mean ḠP gain over all N p patterns in P as: Finally, we compute the Z gain score Z G,p for pattern p as in (10): We sort the patterns in decreasing Then, we successively employ the sorted patterns to adequately cover the normalized gain range, per the sampling criterion of (6).The selected N P * patterns (N P * ⊂ N P ) are used for training and comprise set P * .

E. ELECTRICAL DISTANCE FEATURE
Since d el = d λ , the sampling of the electrical distance feature depends on how the Tx-Rx separation d and the wavelength λ change in the propagation scenarios that the ML model processes.Generating d el samples can be done by changing d (via adding new Tx-Rx pairs), λ (via adding new frequencies), or both.In our case, the goal of generating RSS predictions at the stands of the stadium geometrically restricts the possible Rx positions.Moreover, set A contains pre-defined Tx locations.Thus, we use the second method for sampling the d el feature, varying f for the already selected Tx and Rx locations.Compute Z φ,i and Z θ,i as in (10), using already computed φi , θi , φT and θT .Repeat steps 2-14.

15: end if
We assign a different frequency of operation to each training transmitter, to satisfy the sampling requirement of ( 6) for the desired range of d el values.More specifically, we have that: where d min and d max are the smallest and largest physical distances computed for the Tx and Rx points placed in the stadium, and λ min and λ max are the smallest and largest wavelengths in the training set.The wavelengths are directly computed from the training frequencies.These are uniformly sampled based on a sampling frequency interval f : where i = 0, ±1, ±2, . . ., ±N f /2, f c is the center frequency training frequency of the network, and N f the number of training frequencies.Hence, if the sampling requirement of ( 6) is not met, we increase/decrease f and/or f c , depending on the desired goal.

F. SAMPLING PROCESS
Here we formalize the sampling process by introducing two algorithms.The first algorithm, listed in Algorithm 1, decides the number of training Rx points placed in the stadium and the training Tx-LORs in A * .We start with N R points in the stadium.Based on these, we select Tx-LORs from A according to their Z scores.We check if the Tx selection satisfies our sampling criterion in (6) for each of the θ and φ features.If it does, these parameters are used to train the ANN.If our Tx selection does not satisfy the sampling criteria, we double the number of Rx points and repeat the process.
The second algorithm, listed in Algorithm 2, optimally assigns antenna patterns from pattern set P to the Tx-LORs Compute Z G,p for each pattern p in P, per (13).To compute the required Ḡp and ḠP use ( 11) and ( 12), respectively.3: end for 4: Sort the patterns in P in decreasing |Z G | order.5: Select and add the first pattern to P * , and check sampling requirement of (6).6: while gain sampling requirement not met do 7: Add the next pattern in P to P * and check (6).8: end while 9: return selected P * to train ANN. of A * found by Algorithm 1.This is done by computing the Z scores of the gain for all patterns in P, and by successively selecting the patterns with the best Z gain scores to satisfy the sampling requirement of (6).

V. TRAINING OF THE ANN
We apply the aforementioned sampling process to Stadium 1, to select our training parameters.We place N R = 100 Rx points in the stadium and determine the number of the Rx points, as well as the training Tx-LORs.The algorithm converges after 3 iterations and N R * = 400 Rx points.The Tx-LORs selected by the algorithm based on their Z scores are shown in Fig. 10.Note that we use 6 transmitters (k = 2) per angle feature, for a total of 12.
We then use the selected Tx-LORs in A * to seed Algorithm 2. The algorithm selects 3 antenna patterns in P to be used for training.Their E-plane patterns are given in Fig. 11(a) (pattern 1) and Fig. 12 (patterns 3 and 4).The numbering of the training patterns is in descending order, based on their computed |Z G | scores.For reference, in Fig. 11(b) we show a pattern in P (pattern 2) that is not selected by Algorithm 2. Notice that the selected Eplane patterns offer a wider range of gain values at various φ and θ angles to cover sufficiently the normalized gain range, compared to pattern 2. The same reasoning follows for the H-plane patterns.We assign each of the antennas a different frequency to operate at, with f = 0.5 GHz.This

TABLE 2. Training parameters.
leads to 13 ≤ d el ≤ 3850; the d el range being adequately covered according to the sampling criterion of (6) within a bandwidth of 6.5 GHz.The selected training parameters are summarized in Table 2.
Having selected our training parameters, we use the raytracer to generate the target RSS values associated with the training cases.After performing a convergence analysis on the simulated RSS values of the ray-tracer, a maximum of 1 diffraction, 1 transmission and 3 reflections are used per ray.Then, the RT-generated RSS values are converted to PG values, paired with their corresponding input features and passed on to the ANN for training.When the MAE on the training set is smaller than 2.5 dB we stop training, since after that point we do not observe any noticeable decrease in the validation error (error computed on a 10% subset of the validation set).The trained network is now ready to be evaluated on test cases by computing the MAE between its RSS predictions and the RT-generated RSS values.This is demonstrated in the following sections, by presenting test cases corresponding to Tx configurations employed in actual stadium geometries [14].

VI. GENERALIZATION RESULTS IN A FIXED STADIUM GEOMETRY
The 3D geometry of Stadium 1 can be seen in Fig. 13.There are 51 test transmitters placed around the stadium, above and behind the seating area, pointing towards the center of the   stadium.The vertical tilts for all test transmitters are fixed at θ tilt = 130 • .The transmitters operate at the frequency of 1.99 GHz, not included in the training set of frequencies.All transmitters employ the same antenna pattern, though different from the 3 transmitters used for training.The test E-and H-plane patterns can be seen in Fig. 14.Finally, the test receivers are placed on the stands at different positions from the training ones.The test parameters for the overhead Tx configuration are listed in Table 3.
After using our ray-tracer to generate the test set, we evaluate it on the trained ANN.The results are provided in Table 3.The mean test error is 3.56 dB, while the minimum and maximum test MAE achieved per transmitter is 2.49 dB and 4.11 dB, respectively.
The good agreement between the predictions of the ANN and the actual RSS values is shown in Fig. 15.The figure shows RSS values computed by the RT (on the left) and those predicted by the ANN (on the right), for one of the test transmitters.

A. HANDRAIL TRANSMITTERS
We now study the use of handrail antennas.The Tx configuration can be seen in Fig. 16, while the test parameters of that new case are listed in Table 3.There are 64 test transmitters,  placed directly above and parallel to the seating area, at different heights than before.Note that this Tx configuration is noticeably different than the overhead configuration, especially with regards to the orientations of the transmitters.This leads to significantly different distributions for the generated angle features in the geometry, especially the φ feature.
The transmitters operate at the Wi-Fi frequency band of 2.462 GHz.Their test pattern is also different from the test pattern evaluated previously, as well as the patterns used to train the ANN.The test E-and H-plane patterns exhibit strong omni-directional features, as can be seen in Fig. 17.
After we generate a new test set for the handrail case, we evaluate our trained ANN on it.The test results are presented in Table 3.The trained model achieves a mean test error of 3.22 dB.This is about 0.3 dB lower than the test error achieved for the overhead Tx configuration, validating the generalization abilities of the network, as well as our process for selecting training transmitters.The best test MAE observed for a handrail transmitter is 2.24 dB, while the worst test MAE is 3.72 dB.

VII. GENERALIZATION RESULTS FOR NEW STADIUMS
In this section, we test the generalization abilities of our trained ANN on two new stadium geometries.The first, referred to as Stadium 2, is similar in shape to Stadium

A. STADIUM 2
The first new test stadium is an open-air, single-level stadium.This new stadium is significantly larger than Stadium 1; 34% in length, 44% in width and 60% in height.The test parameters of Stadium 2 are listed in Table 4, while the Tx configuration is shown in Fig. 18.There are 49 transmitters placed in a similar overhead configuration as in Stadium 1.They are placed at different locations compared to the transmitters of Stadium 1, and at θ tilt = 120 • .The transmitters operate at 3.8 GHz, which is not included in the training frequencies.They all employ pattern 4 of Fig. 12(b).
Note that the generated test distribution of the d el feature changes, compared to its distribution for Stadium 1.The new stadium is larger than Stadium 1, and the new test frequency of 3.8 GHz is higher than the test frequency of 1.99 GHz in Stadium 1.However, the sampling of d el at higher training frequencies, up to 6.25 GHz (see Table 2), compensates for that.The test d el distribution is shown in Fig. 19, while the corresponding training distribution was shown in Fig. 7.
We now evaluate the trained ANN on the new test transmitters in Stadium 2. The results are provided in Table 4.The mean test error achieved by the ANN is 3 dB.The improved accuracy is due to the ability of the network to generalize to the new geometry, even though Stadium 2 was not part of its training set.The worst test MAE achieved by a transmitter in Stadium 2 is 3.56 dB, while the best test MAE is 2.44 dB.

B. STADIUM 3
We now increase the complexity of the test geometry by using a multi-level open stadium.Its geometry can be seen in Fig. 20 We employ again our trained model and evaluate it on the data generated by using the 8 transmitters of Stadium 3 for the two test patterns.The results are presented in Table 4; for the overhead pattern the results are listed in parentheses.The test error of our trained model is 2.51 dB for the original test pattern.Note that this error is about 1 dB lower than the test error observed in Stadium 1, despite the fact that our test geometry is significantly different from Stadium 1.The best and worst test MAEs achieved are 1.71 dB and 2.79 dB, respectively.The test MAE for the overhead pattern increases to 3.45 dB, due to the more complex test pattern.A similar increase is observed for the best and worst test MAEs achieved, at 2.21 dB and 3.83 dB, respectively.

C. DISCUSSION
Designing a generalizable ML-based propagation model requires the selection of helpful input features.These should capture the influence of the parameters we want to generalize  with respect to, on the output quantity of interest (RSS in our case).The next requirement is to sample these features efficiently.We presented a strategy for doing that in Section IV-F using two algorithms.These algorithms worked well in our case, but the main idea is general.We need to adequately and efficiently sample the training input features to do well on a range of possible test features.
Regarding the algorithm itself, the selection of Tx-LORs depends on the sampling criterion of (6).Since training Tx-LORs are selected from the validation set A, we have to ensure that sufficient number of candidate Tx-LORs are included in it.The same applies for pattern set P; it should have a sufficient number of diverse antenna patterns to meet any required sampling criteria for the gain feature.
The ANN model takes approximately 26 • t RT to train, where t RT is the runtime for the RT simulation of a single transmitter and 400 receiver points in the training stadium geometry.This time involves both the data generation cost for simulating the training antennas in the ray-tracer, the cost for processing the simulated data (loading the saved data and computing the input features), and the cost for training the ANN model by minimizing its cost function.Thus, the more simulations the trained model is required to run, the larger the time savings are, compared to running the same simulations on the ray-tracer.That is easy to achieve in modern stadiums, especially for tasks such as optimizing the placement of transmitters, where hundreds of candidate Tx locations have to be simulated, often at a variety of different frequency bands.
Finally, our algorithm can be easily calibrated with a small dataset of measured data, especially for new stadiums.These can be used in a similar fashion as with the ML technique of transfer learning [24].In such a case, a new output layer replaces the existing output layer of the trained ANN.The already learned weights of the network are fixed.Then, the network is trained for a small number of iterations on the subset of measured data, by only updating the weights of the new output layer.

VIII. CONCLUSION
In this paper, we presented a ML-based model for radiowave propagation in stadiums.We outlined the workflow of our model, an ANN, as well as its input and output.Regarding the input, we explained what input features to use and how sample them to create a highly generalizable model.That generalizability is not achieved by exhaustively training the model with large amounts of data, but rather guided by careful selection of optimal input parameters.These parameters include the locations, orientations and antenna patterns of the transmitters, that help the network learn more efficiently.Our sampling algorithm apart from novel is also general, since it can be used in any environment by adjusting the sampling criteria and ranges of the input features.
The output of the ANN (RSS values at specified points in the stadium) was evaluated for a variety of different realistic propagation scenarios, including various antenna patterns, Tx-Rx placements and frequencies, in a variety of stadium geometries.In all cases, the network generated accurate RSS predictions proving its generalization abilities.Our MLbased propagation model is the first such general model that can be used for propagation in stadiums.It also combines high generalizability with computational efficiency, since predictions for new test cases can be generated very fast.The proposed model can be incorporated in the design of communication systems in stadiums, and for evaluating and optimizing the positions of transmitting antennas.Therefore, it advances the state of the art in an area of significant interest for radiowave propagation studies.

FIGURE 1 .
FIGURE 1. Overview of the data generation process.The ray-tracer is first configured by setting a number of parameters relating to the propagation problem (e.g.geometry, location of Rx points etc.), as well as its function (e.g.number of ray interactions allowed).The solver then outputs the RSS at the specified Rx points.The geometry of the stadium has to be converted into a facetized form to be used by the RT solver.Such a form is shown on the right, where a geometry rendering of a stadium has been generated in the Blender computer graphics tool.Stadium image source: https://www.ostadium.com.

FIGURE 2 .
FIGURE 2. The input features of the ANN.Note that the (x, y, z) coordinate systemshown is the local system of the transmitter.

FIGURE 3 .
FIGURE 3. The workflow of the ML model.

x x x 4 FIGURE 4 .
FIGURE 4. The architecture of the ANN.

FIGURE 5 .
FIGURE 5.The training and test process of the ANN.

FIGURE 6 .
FIGURE 6. 2D view of Stadium 1.We show the two bounding boxes defining the rectangular frame that encloses the geometry.

FIGURE 7 .
FIGURE 7. Two of the generated training feature distributions: (a) del distribution (b) φ distribution.

FIGURE 8 .
FIGURE 8.A transmitter exhibiting φtilt with respect to the x axis, and another with aθtilt with respect to the z axis.Note that, contrary to the coordinate system in Fig.2, the coordinate system shown here is the global coordinate system.

FIGURE 9 .
FIGURE 9.An example of Rx placement in Stadium 1.

Algorithm 1 3 :
Training Rx and Tx-LOR Selection 1: Initialize N R ← 100 2: for i = 1 to N T do Compute φi and θi for Tx-LOR i in A, per (8).4: end for 5: Compute φT and θT over all Tx-LORs in A, per (9).6: for i = 1 to N T do 7:

8 : end for 9 :
Form a training set by selecting the 3k Tx-LORs (k = 1, 2) with the N k best and N 2k worst Z scores per angle feature.10: if sampling requirements per (6) are met for some k then 11: return selected A * to train ANN.12: else 13: Double N R .14:

Algorithm 2
Training Pattern Selection 1: for p = 1 to N P do 2:

FIGURE 10 .
FIGURE 10.The Tx placement in Stadium 1 that is used to train the ANN.The main beams of the antennas align with the direction the arrows point to.

FIGURE 11 .FIGURE 12 .
FIGURE 11.Two examples of E-plane antenna patterns in P; (a) Pattern 1 (used for training) (b) Pattern 2 (not used for training).The θ = 90 • direction is along the axis of the antenna.

FIGURE 13 .
FIGURE 13.Stadium 1 geometry.Transmitters are shown in red cones.Dashed line encloses the stadium section used for the RSS comparison of Fig. 15.

FIGURE 14 .
FIGURE 14.The (a) E-plane and (b) H-plane pattern of the overhead test antennas in Stadium 1.

FIGURE 15 .
FIGURE 15.RSS comparison between the target RSS values on the left, and the predicted RSS values on the right, for one test transmitter (noted with red).

FIGURE 16 .
FIGURE 16.Tx configuration for the handrail setup in Stadium 1. Transmitters are shown in red cones.

FIGURE 17 .
FIGURE 17.The (a) E-plane and (b) H-plane pattern of the handrail test antennas in Stadium 1.

FIGURE 18 .
FIGURE 18. Geometry of Stadium 2. Transmitters are shown in red cones. 1 but considerably larger.With this test case we focus on whether our training d el distribution ensures the accuracy of the ANN on stadiums with significantly different dimensions from our training stadium.The second stadium, referred to as Stadium 3, poses a more difficult case, having multiple seating levels, in contrast to the single-level architecture of Stadium 1.This case tests if the trained network can generalize to significantly different geometries.

FIGURE 19 .
FIGURE 19.Mean del test distribution for Stadium 2.
(a).The stadium consists of 3 seating levels and is larger than Stadium 1 by about 18% in length, 23% in width, and 265% in height.The test parameters of Stadium 3 are listed in Table 4.There are 8 overhead transmitters around the stadium, at vertical tilts of θ tilt = 110 • .The transmitter orientations on the azimuthal plane are shown in Fig. 20(b).The antennas operate at the frequency of 1.99 GHz.The original pattern used by the test antennas different from the ones used to train the ML model in Stadium 1, as can be seen in Fig. 21.We also use the test pattern of Fig. 14 of the overhead antennas in Stadium 1, as an additional test pattern for Stadium 3.

FIGURE 20 .
FIGURE 20.Stadium 3 configuration: (a) the geometry and (b) the Tx configuration.

FIGURE 21 .
FIGURE 21.The (a) E-plane and (b) H-plane pattern of the test antennas in Stadium 3.