An Empirical Assessment of the Contribution of 5G in Vertical Industries: A Case for the Transportation Sector

5G has been heralded as a critical, decisive technology for verticals’ digital transformation. It targets innovative technical capabilities, allowing dedicated and improved communication mechanisms, going beyond the mere performance increase experienced by past generations’ evolution. Despite the literature already envisaging the application of 5G-enabled solutions in vertical-based deployments, there is still a lack of experimental insights shedding light on the actual feasibility of 5G capabilities in specific vertical deployments. This gap is particularly evident in verticals that did not traditionally rely on mobile network solutions for their communication needs, such as safety-related control signals in railway operations. This paper aims to fill this gap by delivering an extensive empirical analysis performed through a 5G-enabled railway deployment. We designed and implemented a joint testbed, which allows us to faithfully reproduce two important transportation railway use cases that leverage 5G communications involving the vertical, communication service solutions operator, and integration actors. There, we expose the necessary integration engineering behind the enablement of 5G capabilities in such scenarios, considering the highly safety-certified and standardized environment of railway vertical sector regulations (which never considered the utilization of mobile networks in their development). We empirically demonstrate the performance capability achieved by the 5G-enabled network using two different systems, a carrier-grade one and an open-development one, in light of the required key performance indicators. To the best of our knowledge, this is the very first field trial and empirical evaluation of the suitability of 5G with the implementation of a 5G-enabled railway cross-level scenario within the transportation context.


I. INTRODUCTION
The fifth generation of mobile networks [1] (5G) stands out from previous generations by establishing new additional The associate editor coordinating the review of this manuscript and approving it for publication was Bilal Khawaja . service delivery models, whose flexibility and dynamics (allied with increased performance) are tailored towards industry sectors and verticals [2]. These underlying characteristics have manifested in its ability to provide enhanced mobile broadband (eMBB), massive Machine Type Communications (mMTC), and Ultra-Reliable Low-latency communications (URLLC) services, distinctively through the existence of network slices [3]. For industrial sectors' stakeholders already deploying some form of connectivity via mobile networks, such ability allowed the improvement of existing communication scenarios and paved the way for developing new ones (i.e., unable to be fulfilled by previous generations) [4]. However, it also presented itself as a beacon of opportunity to, for the first time, fully integrate mobile communication capabilities into other sectors' operational processes.
There are many challenges associated with the interaction between different industrial domains for the first time. These challenges range from different innovation cycles, requirements associated with certification for safety and security, non-interaction between standardization bodies, and even a lack of expertise and knowledge on the opposing industrial domain.
Entities such as the 5G Alliance for Connected Industries and Automation (5G-ACIA 1 ) have brought together organizations from both operational technologies and information and communication technologies (ICT), intending to smooth out the application of 5G in industrial deployments. This objective is also pursued by other entities that address specific operational sectors, such as the 5G Automotive Association (5GAA 2 ), the Alliance of Industrial Internet (AII 3 ), and the Industry IoT Consortium (IIC 4 ). Additionally, research projects such as H2020 5G-enabled Growth in Vertical Industries (5GROWTH 5 ) aimed to pursue the technical and business validation of 5G technologies, creating the necessary infrastructure and technical facilities to support the required integration engineering of verticals for 5G up taking.
Under the scope of that project, this paper presents the engineering story and technical lessons learned from deploying 5GROWTH-enabled 5G mobile mechanisms in the scope of the communications idealized with two transportation use cases related to the railway sector. The paper aims to provide a two-fold contribution: on the one hand, it serves as a reference towards verticals (especially of the transportation sector) by not only showcasing the identified opportunity for coupling 5G communications but also addressing the required engineering processes for its adoption, and the Key Performance Indicators (KPIs) measurements obtained; on the other hand, it presents the communications infrastructure, the journey from the laboratory to the field trials, and a testament towards ICT stakeholders (e.g., vendors, operators, and integrators) of 5G's underlying capabilities, flexibility, and limitations withstanding for adopting verticals' cases.
The remainder of this article is structured as follows. Section II provides a technical perspective of the underlying platform used to provide 5G to the transportation vertical, highlighting the main concepts of 5G (II-A), the 5GROWTH project (II-B), and the actual infrastructure used (II-C). Section III introduces the addressed transportation vertical use cases and determines the respective KPIs (III-B). Section IV introduces the different trial facilities and test methodologies. The laboratory 5G performance results are presented in section V, followed by the field trial results in section VI. Section VII critically assesses the results, and section VIII concludes the paper. Finally, to support the reading of the paper, we have also added a list of acronyms and abbreviations in section IX.

II. NETWORKING TECHNOLOGICAL LANDSCAPE
The networking technological landscape addressed in this paper is comprised of three main areas that will be detailed in the following subsections: (A) the 5G mobile networks, (B) the H2020 5GROWTH project, and (C) the deployment infrastructure.

A. 5G
Previous generations (up to 4G) focused on data rate increase for smartphone usage by applications (primarily targeting human end users). 5G turns its attention to new scenarios involving devices and the existing end-users, considering the necessary technological aspects for heterogeneous communication requirements and different scales of connected entities. This evolution allows vertical industries to improve not only obtained service performance but also their flexibility, considering different criteria such as latency, reliability, availability, resilience, and density, amongst others, using a single access technology at the last hop [5]. In order to achieve this, 5G had to procure evolutions not only at the radio transmission level or even the core network itself but also integrate key innovative architectural aspects, such as cloudification. Specifically, 3GPP adopted a microservice-oriented Service Based Architecture (SBA) for the 5G core [6], pointing towards a cloud-native solution deployment [7], supported by mechanisms such as Software Defined Networking (SDN) and Network Function Virtualization (NFV) [8], thus enabling scaling microservice instances and placing those instances on available cloud servers. By combining these aspects with 5G's New Radio (NR) ability to allow the dynamic allocation of bandwidth partitions to offer different classes of traffic [9], network slicing allows for differentiating network service provisioning towards different devices and users in the same coverage area [10]. This ability is of great importance to verticals, as their traffic needs to be isolated from other data flows for different reasons, such as performance [11] and security [12]. Additionally, 5G also brings new possibilities that benefit existing scenarios and technology directions, such as Cellular Vehicle to Everything (C-V2X). These enhancements, compared to what LTE provided for C-V2X, allow for the support of more specific and demanding scenarios in vehicular networks by being able to provide URLLC and unicast and groupcast modes, enabling use cases that were not possible with previous C-V2X generations (e.g., platooning). A table summarizing such enhancements is provided in [13]. In addition to the radio layer enhancements, several architecture enhancements in 5G [14] are introduced such as specific C-V2X network slices and QoS parameters and the introduction of the V2X Application server as an Application Function (AF), allowing it to interact with the 5G network in order to influence its behavior, provision parameters via the Policy Control Function (PCF) and gather monitoring data.

B. H2020 5GROWTH
The enhancements provided by 5G not only had the capability to incrementally enhance and evolve current usage scenarios, but was also developed to target its deployment in new types of vertical environments. In some of these segments, industrial stakeholders were complete newcomers to the prospect of mobile networks. Aspects such as completely different technical and technological environments between the telco and industrial segments, were considered hard barriers for a smooth integration of 5G technologies in verticals. Therefore, it was crucial to develop a joint effort that brought together the different elements of the business chain, in order to determine the necessary business cases, technological developments, integration and validation efforts.
In this respect, the 5G Infrastructure Public Private Partnership (5G-PPP 6 ) was an European initiative built to support the development of ICT communication networks and infrastructure towards 5G and its adoption within the industry under the Horizon 2020 research program. Its research agenda was composed of three distinct timely phases: the first ranging from 2015 to 2019 and aggregating the first wave of research and innovation projects. The second, from 2017 to 2022, targeted the optimization of initial results. Finally, the third phase, from 2018 to 2024, emphasizes large-scale trials and results up taking. 7 The 5G-PPP is also composed of a set of different working groups, ranging from actions related to architecture, prestandardization, spectrum, SDN/NFV, network management and Quality of Service (QoS), vision and societal challenges, security, small-medium enterprises (SME), trials and connected car.
5G-PPP's third phase had its projects structured into different parts, with outcomes from one part able to be fed into the work of forthcoming parts. These six parts individually addressed projects focusing on infrastructure, automotive, advanced 5G validation trials across multiple vertical industries, 5G Long Term Evolution (LTE), 5G core technologies innovation, and 5G for connected and automated mobility (CAM) and finally, 5G innovations for verticals with third party services and smart connectivity beyond 5G. 6 https://5g-ppp.eu/ 7 Presented dates are not officially established deadlines but the result of analyzing the start and end times of projects belonging to each phase. This analysis considers, for example, projects which have been given time extensions, e.g., due to the COVID-19 pandemic.
5GROWTH was a research project belonging to Part 3 (i.e., advanced 5G validation trials across multiple vertical industries) of 5G-PPP's Phase 3, intending to explore the concrete applicability of 5G technologies to real-world use cases across various vertical sectors. Concretely, it addressed the technical validation of using 5G technologies with verticals from the Industry 4.0 (I4.0), Transportation, and Energy sectors, working out the necessary technology integration considering the specific challenges of each particular operational sector. The project featured trials in Spain (related to I4.0, featuring the remote operation of a connected worker performing operations over quality assessment equipment), in Italy (also related with I4.0 featuring a digital twin use case allowing a factory plant manager to receive live information of the production line through a digital representation of the factory), and in Portugal (one pilot featuring an energy sector advanced monitoring and maintenance solution for distribution stations, and a second pilot featuring railroad level crossing communications for the transportation sector). This is the motivation for this paper: to provide the experimental validation conclusions of exposing a railroad industry vertical to the capabilities provided by 5G, taking into consideration the requirements and specifics of utilization scenarios under such an environment. These conclusions were obtained under the scope of the three-and-a-half year span of cooperation between several partners addressing this use case in the Portugal railroad pilot of the H2020 5GROWTH project. Considering Table 1, which analyzes other research results under scope of different supporting frameworks, our paper showcases a unique vertical scenario, exploring both a 5G carrier-grade and a 5G experimental deployment, first under a laboratory capability (i.e., indoor), and then on the field (i.e., outdoor). The aim is to generate a bilateral learning outcome, of integrating novel telco-based capabilities in a vertical industrial segment, improving both ends with important lessons and insights.
For the Portuguese railroad case in the 5GROWTH case (which is the one pursued in this paper), we emphasize three key stakeholders: first, EFACEC is a Portuguese company that develops Energy, Environment, and Transportation engineering solutions. For the specific case of the validation conducted in this paper, EFACEC produces turn-key solutions for railway level crossing operation, automatically detecting incoming trains and lowering barriers to stop traffic and pedestrians, according to the necessary worldwide established standards and certifications. Second, Altice Labs composes the research and development branch of the Altice group, which also includes one of the biggest Portuguese telecommunications operators. In the context of the validated use cases, it is interested in developing onpremises 5G mobile connectivity solutions and partnerships. Finally, the Instituto de Telecomunicações (ITAV), a national research unit situated on the campus of Aveiro University, aims to provide integration expertise and consultancy to all the technical domains involved in the evaluated scenarios while driving the necessary research.
One crucial consideration from 5GROWTH is that it leveraged the resulting infrastructure of 5G-PPP's Phase 3-Part 1 projects (i.e., Infrastructure), as the underlying base for the 5G mobile network platforms used for 5GROWTH's trials. These projects also commonly known as ''ICT-17'' projects (as they answered the 17th funding call topic of Horizon 2020's ICT program between 2018 and 2020) included H2020 5G EVE 8 and 5G-VINNI, 9 integrating the Spain and Portugal trial sites, respectively.

C. DEPLOYMENT INFRASTRUCTURE
The deployment infrastructure herein described is separated into two subsections due to the two different 5G network infrastructures used in this article. The first is an ICT-17 platform resulting from the H2020 5G-VINNI project. The second is a carrier-graded infrastructure resulting from the 5GAIner partnership.
1) ICT-17 PLATFORMS: H2020 5G-VINNI 5G-VINNI (5G Verticals INNovation Infrastructure) is an ICT-17 project from 2018 to 2021. It aimed to accelerate the 5G adoption in Europe by providing an end-to-end (E2E) infrastructure that vertical industries could use to develop and validate their pilots, thus lowering the entry barriers and fostering 5G take-up. Under this project's scope, several 5G facility sites were designed and built in Europe. These sites comprised 5G radio and core components, coupled with management and orchestration tools, allowing for the validation of 5G KPIs in different testing campaigns, as well as supporting the execution of vertical trials coming from subsequent projects (such as 5GROWTH), thus allowing 5G-VINNI's lifetime to be extended beyond the project's original time-frame.
One of 5G-VINNI's experimental facilities is in Aveiro, Portugal (where the 5GROWTH's Transportation trial also takes place). It is based on an extensive computational and networking infrastructure available in ITAV and deployed by Altice. Under the scope of 5G-VINNI, besides designing, implementing, and integrating the infrastructure, experimentation activities were carried out over it in regards to NG PON2-based 5G front-hauling and backhauling, Operations Support System (OSS) network slice life-cycle management, and edge computing.
For 5GROWTH, particularly the Transportation vertical trial presented in this paper, the 5G coverage of 5G-VINNI was deployed over the Aveiro Harbor area as an extension to the existing experimental site in Aveiro, which encompassed the ITAV premises.
The infrastructure, showcased in figure 1, contains a deployment of the SONATA 10 Management and Orchestration (MANO) Service platform, which has been deployed in other research projects such as H2020 5GTANGO. 11 It consists of several components running as microservices and interacting with each other to manage the lifecycle of Virtual Network Functions (VNF), Network Services (NS), and slices. It implements an ETSI NFV Orchestrator (NFVO) and VNF Manager (VNFM) [37], responsible for onboarding and managing the lifecycle of VNFs and NSs.
For 5GROWTH's Transportation trial, SONATA is responsible for deploying and providing a 5G core, namely Fraunhofer's Open5GCore 12 Release 15. This trial is deployed on a blade server with two 12-core CPUs, a total of 256GB RAM, two 480GB disk SSD, two 2TB SAS 7200rpm SFF hard drives, and two 2-port 10Gbps Ethernet cards.
This 5G core will be connected to two stages of the trial's RAN. The first stage considers an initial deployment of the RAN at ITAV premises. The final stage considers the deployment of the RAN at the Aveiro Harbor. Both stages consider the availability of a fully functional 5G infrastructure, with RAN and core.
The RAN component used in the trial site is based on ASOCS CYRUS 2.0, providing a virtualized open-software cellular solution. It is based on the Open RAN architecture [38], and it is divided into several components, following the 3GPP 5G RAN Stand Alone (SA) architecture [39], namely the Centralized Unit (CU), Distributed Unit (DU), and Radio Unit (RU).
The CU and DU elements are connected to the Open5GCore. A cross-haul NG PON2 connects them to a remote RU, which serves end users. The end-user part is provided via a Huawei 5G Customer Premises Equipment (CPE) Pro H112-372, which can connect to devices via Wi-Fi and Ethernet. A 5G test frequency with 100MHz bandwidth was used, provided by a Portuguese mobile operator. The technical specifications of the ASOCS equipment are shown in Table 2.
The end user CPE is connected to Open5GCore through its N1 interface (which terminates at the Access and Mobility Management Function (AMF)). The AMF is also the termination of the N2 interface, which connects it to the ASOCS RAN components (i.e., RU, DU, and CU). These components also connect Open5GCore to the User Plane Function (UPF) in the realization of the N3 interface [6].
It is essential to highlight that ASOCS constituted a 5G evaluation kit and, therefore, not a final commercial product. However, it provided the ability to assess the vertical trials in light of an innovative 5G architecture (i.e., O-RAN). Additionally, it also allowed important feedback regarding the development of a new 5G-based mobile network solution by the communications service solutions provider, which further increments the exploration relevance of the trial conducted at the 5GROWTH's Aveiro trial side, in addition to the vertical's own expectations perspective as a 5G solutions consumer.   Finally, for the particular case of the final phase trial (i.e., in the Aveiro Harbor), an external antenna was deployed on top of a water tower, providing a line of sight to the railway level crossing, train detecting sensors, and the trains themselves.

2) 5GAIner INFRASTRUCTURE
In addition to the 5G-VINNI deployment, 5GAIner 13 was the second available infrastructure that provided an alternative solution at the same premises with coverage points at ITAV 13 https://5gainer.eu/uc-transport.html and the Aveiro Harbor. Test frequencies were provided by the same operator as the deployment presented in section II-C1 but featured carrier-graded equipment instead.
This carrier-graded infrastructure from 5GAIner is based on a commercial 5G solution from Huawei that encompasses outdoor and indoor radio cells, together with a 5G SA Core (Release 15 when the experiments were conducted) and a dedicated Multi-access Edge Computing platform. This infrastructure is geographically distributed, as shown in Figure 2, in four different locations: (i) two on-campus indoor deployments; (ii) one off-campus outdoor deployment; and (iii) one off-campus indoor, edge-based deployment.
The 5G SA Core is deployed on two E9000 blade server chassis equipped with three management nodes, and 18 compute nodes, which accounts for a total of 42 CPU and 5.25 TB of RAM. This computational power is complemented by two dedicated OceanStore 5300 v5 storage units, with a total capacity of over 140TB. The networking follows a standard datacentre deployment with two top-ofrack (ToR) switches, two end-of-rack (EoR) switches, and two datacentre gateways. This networking fabric is composed of 4 CloudEngines (layer 2 operations) and two NetEngines (layer 3 operations). The NFVI runs on top of this hardware, powered by FusionSphere and FusionStage, which in turn  hosts the different functions of the 5G SBA. The deployed 5GC has the following functions available: AMF, SMF, AUSF, NSSF, NRF, UDM, and UPF. This infrastructure is monitored using Huawei eSight platform and a Mobile Automation Engine (MAE) solution. This last element also assumes the orchestration and life-cycle management mantle for the 5GC functionalities.
The work presented in this paper was conducted in two of these locations: • The preliminary experiments aiming at initial concept validation and assessment were conducted in one of the indoor sites deployed in ITAV premises which consists of two antennas (pRRU 5961), a BBU 5900, and an RHUB 5963 that makes the connection between the BBU and the antennas; • The real-world deployment utilized for technical and functional validation was conducted at the outdoor site in the Aveiro Harbor, 6km away from the university campus, composed of three outdoor antennas (AAU 5649) placed on top of a tower and covering three different sectors. The AAUs are connected to a BBU 5900, which is connected to the 5GC through a public L2 service provided by Altice. The radio and power configuration of the 5GAIner network for the different experiments are summarized in Table 3.

III. TRANSPORTATION VERTICAL
Wireless communications are fundamental in many applications in the transport sector where, for obvious reasons, wired communications incur enormous construction costs or cannot even be implemented. 5G associates the flexibility of wireless communications with traffic isolation resulting from network slicing and brings performance advantages such as low latency and increased bandwidth [40].
This section describes two illustrative use cases for exploring 5G capabilities in the operational procedures of railway cross-level communications from a Transportation sector's vertical. Additionally, it also describes the considered KPIs, as well as their deployment over the test facilities.

A. USE CASES
Here we will present two use cases, one related to safetycritical communications, where low latency is critical, and another related to non-safety-critical communications, where bandwidth is more relevant. The first use case uses the 5G network to connect the train crossing detection sensors to the level crossing controller. The second use case uses the 5G network to transmit live video streaming of the level crossing into the train when it approaches the level crossing. The geographical deployment of the trials took place in the city of Aveiro, Portugal, more specifically in the Aveiro University Campus and the Aveiro Harbor, as shown in figure 3. Figure 4 presents the site deployment of this use case. For this trial, a single-line heavy freight train rail branch was used, which serves the Aveiro Harbor, connecting it away from the main national railroad. This 9-kilometer long branch (i.e., 5.6 miles, approx.) has a low frequency of freight trains, which made it ideal for the trial tests in our deployment in order not to interfere with the regular operations of the trains. Close to its final destination in the port, the rail is crossed by a road featuring a conventional automated level crossing.

1) USE CASE 1
As shown in figure 5, some kilometers away from the level crossing itself, in both directions, there are strike-in detection sensors that can detect the individual axis of the trains. This information is sent towards a Level Crossing (LX) Controller, which, upon detecting a freight train axles, commands the barriers in the road to lower and stop vehicles. The LX Controller counts the number of train axles that pass VOLUME 11, 2023  through the first strike-in detector. As soon as the same axel count has been detected passing through the opposite strikein detector, the LX Controller commands the raising of the barriers, and the vehicles can safely pass through.
In a conventional level crossing, the train crossing detection sensors are directly connected to the LX Controller through a dedicated cable link (i.e., as the current line of EFACEC products has in the market). Depending on the train's speed, these sensors for detecting the train's passage may be a few kilometers away, which requires the opening of large trenches to deploy the cables below the ground, thus avoiding tampering or weather-related wear. As a result, this method for enabling communications between the devices imposes high construction work costs and prevents any dynamic infrastructure reconfiguration or planning.
This use case is directly related to the system's intrinsic security, and it uses the 5G network for critical communications. As such, the objective is to eliminate these interconnection cables and the need to open these vast trenches by using a 5G network and solar energy. Thus interconnecting these sensors allows the detection of passing trains and sends the information to the LX controller. A detailed assessment of the contributions of 5G into the transportation sector is provided in [4], which presents a business analysis considering the European market.
To implement this use case, the prototype equipment represented in the diagram of figure 6 was used. In the center of the figure, we have the equipment that is deployed closest to the LX Controller. This equipment consists of: first, a Quectel RM500Q 5G modem connected via a USB3.0 M.2 converter to a Raspberry Pi, in turn, responsible for housing the necessary Linux scripts that connect to the 5G network and creating the necessary secure IP tunnels. Second, a Frausher IXL train counter hardware is responsible for aggregating and processing the information from both sensors to where the LX Controller will be connected. On both sides of the figure, we represent the equipment that is also physically deployed on both sides of the level crossing. This equipment consists of a similar 5G CPE used in the LX controller, the Frausher COM-FSE, and the train detection sensor. The 5G-CPE is responsible for connecting to the 5G network and establishing secure tunnels that connect the sensors to the LX Controller. The tunnels established between the three 5G-CPEs are OpenVPN tunnels. The Frausher COM-FSE is a hardware device that ensures the interconnection to the actual train detection sensors, gathering information such as radius, speed, number of axles, and others, and communicating it to the Frasusher IXL.

2) USE CASE 2
This use case exploits the potential of 5G's capability to provide a safety-contributing feature that complements the previous mechanism. In this use case, the intention is to send the approaching train a live video footage of the next approaching level crossing, as shown in figure 7. This way, the safety conditions will be higher when it passes through the level crossing, especially when the train driver has to see the level crossing to understand its current situation. It is important to note that, with the current railway-based communication systems (i.e., without 5G), this use case is not possible and thus manifests an essential opportunity for railway solution providers. Figure 8 presents the site deployment of this use case. The concept considers that the level crossing has video surveillance capabilities on-site, allowing for the collection of live footage from what is currently happening in the place. On its turn, the train is equipped with a screen allowing the conductor to visualize the video surveillance footage of the next level crossing before the train reaches it. Such information can also be sent to other destinations, such as maintenance crews operating on the rail track (i.e., through a tablet or smartphone), to verify conditions on nearby level crossings and ensure the safety of the workers.    The equipment used to implement this use case is represented in figure 9 and is described as follows.
On the right side of the figure, we have the equipment located next to the level crossing. The Huawei CPE5G Pro1 5G CPE router, a Raspberry Pi, and an IP video camera were deployed here. The CPE router allows access to the 5G network, which assigns an IP to its WAN interface. On the LAN side of the router, we connect the video camera that will allow capturing the image of the level crossing and a Raspberry Pi that runs a VPN Wireguard 14 server, which will allow establishing a secure tunnel to the equipment on-board the vehicle and also does NAT for the local network. On the left side of the figure, we have the equipment on board the vehicle. Here we have another 5G CPE router and a Syslogic PC console. The 5G router will allow access to the 14 The different security alternatives in each use case are to provide solution diversity in the event a vulnerability is found. Even though it is not the main topic of this paper, the vertical railroad solution provider wanted to evaluate the performance impact of the two different solutions: initially, it started with OpenVPN, but after evaluating Wireguard with overall better performance and stability, Wireguard was preferred for our field assessment. 5G network and connects the Syslogic PC console to the LAN side. The Syslogic PC console, with Debian Linux installed, runs a video application showing the surveillance footage in real time. It is also where the Wireguard client secure tunnel is created, towards the remote level crossing.

B. KEY PERFORMANCE INDICATORS
The main target for the first use case is to achieve a low latency allowing the support of safety-critical communications. In contrast, the second use case's main target is guaranteeing a data transfer at a high data rate required for video transmission in scenarios involving mobility. Therefore, the first use case can be considered a URLLC application, and the second an eMBB application.
In the use case concerning critical communications, the most relevant KPIs are latency, jitter, and packet loss. Crucially, the latency must be less than 20ms to comply with railway signaling safety communications standards [41], thus assuring a Certified Level crossing. Jitter and packet loss should be as low as possible not to affect a safety algorithm that exists in level crossings, which will automatically enter them into a safe mode (i.e., the road barriers are lowered in precaution). This safeguard would happen whenever there is communication loss or out-of-sync between the communication components, blocking road traffic. This situation has a massive impact on operation, can motivate penalties, and requires the presence of maintenance people to recover from this operation mode manually. Therefore, the jitter should be less than 5ms and packet loss less than 0.05%. Bandwidth is irrelevant given the low data volume involved; however, the safety standards mandate a 10 Mbps data rate for protocolar messages.
In the second use case, bearing in mind that it is intended to transmit real-time HD video, the network needs to ensure the appropriate resources to do so with quality and without delay. That means a sustained bandwidth, low end-to-end latency, and jitter, as per Video Surveillance Systems standard [42]. The latency should be as low as possible, aiming at a total latency of less than 100ms. This total latency is composed of several components, among which is the latency of the transmission channel in the 5G network, which should not exceed 20% of the total value. Jitter should be as low as possible since to cancel out the influence of jitter requires buffers in the reception and, as such, will lead to an increase in latency, compromising the required less than 10ms. Simultaneously, the guarantee of a relatively high bit-rate is also needed, as it is intended to transmit Full HD video with a low packet loss rate (i.e., lower than 0.1%), not to compromise the quality of the video (i.e., motion clarity, picture detail, and low delay). In the trials, as the railway was composed of a rail branch for freighter trains passing through a level crossing close to its final destination, we had a slow-moving vehicle. In this sense, Youtube Live streaming Full HD @60Hz would require 4.5 Mbps to 9 Mbps. 15 In our scenario, sustaining 16 Mbps should ensure the video quality while giving enough leeway to cope with abnormal network events (e.g., the < 0.1% allowed packet loss). The 80 Mbps bit rate up to 60 Km/h estimated by the safety communication standards for wired communications was not considered a priority as the newer and more efficient codecs demand less bandwidth while still being resilient to small packet losses. Table 4 summarizes the performance requirements for both use cases.

IV. TEST FACILITIES AND METHODOLOGY
The most significant amount of tests focused chiefly on use case 2 (i.e., Wireguard was used), but the findings are representative of both use cases' conclusions. These tests were carried out initially in a laboratory and then in a real environment at the level crossing next to the Aveiro Harbor. In this last case, the 5G radio unit was placed on top of the water tank, at around 650m from the level crossing (approx. 2132 ft), as well as 547m (1794 ft) and 840m (2755 ft) from the closest and furthest axle sensor, respectively.

A. ITAV LABORATORY
The test environment used in the ITAV laboratory is similar to the one used in the real environment except for the location mechanism, where GPS is used in the real environment, but the location is emulated in the laboratory. The equipment used in this test environment is described in section III-A2. Ping and iperf3 16 tests are run on the Syslogic console, with the traffic going through the secure Wireguard tunnel on the 5G network to the other end, namely the Raspberry Pi where the iperf3 server is running. The application is running on the Syslogic console, based on location; as it approaches the level crossing, it will create the Wireguard tunnel, which is used to request the video stream. These tests were realized in two phases: we first tested an initial version of Altice's 5G SA solution integrated into the 5G-VINNI framework, helping its iterative improvement; in a second phase, we tested the 16 iperf3 network bandwidth measurement tool -https://github.com/ esnet/iperf final version of Altice's solution and, additionally, a Huawei commercial-based solution belonging to the 5GAIner project.

B. AVEIRO HARBOR PILOT
The test environment used at the Aveiro Harbor is similar to the one used in the laboratory described above, except for the location mechanism, which is now obtained by GPS instead of emulated. Here we tested Altice's 5G SA solution integrated into the 5G-VINNI framework, offering coverage at the Aveiro Harbor area and the 5GAiner deployment, with coverage in the same area of the Aveiro Harbor.

C. METHODOLOGY
We must assess if latency, jitter, throughput, and packet loss remains within the set KPIs (section III-B).
The latency evaluation relies on 5000 ICMP request-replies (i.e., ping) that measure the RTT end-to-end between the Syslogic PC console and the Raspberry Pi at the other end. Even though 5G radios are usually asymmetric in nature (i.e., uplink is different from downlink), our tests consist of two 5G CPEs contacting each other. The ICMP packets will invariably go through the same two up-link hops and two downlink hops with similar radio characteristics in every request-response; thus, the route is symmetric. The symmetric route allows an excellent estimation of the one-way latency as half of the measured RTT. This estimate provides crucial insight into the performance when used with one-way signaling (such as in UC1). We must note there is an encrypted tunnel between the CPEs, being the evaluation carried out with Wireguard tunnels.
The same latency tests can be used to calculate the jitter without network load. However, we have preferred highlighting the jitter under network load as reported by the iperf3 tool. This preference is because jitter under load is more suited to evaluate UC2, where the camera video stream may put a substantial load on the uplink. Furthermore, the iperf3 tool will measure jitter using UDP packets, the same transport protocol used for communications in both use cases (e.g., Wireguard and RTP).
We have also used iperf3 to assess the bandwidth and packet loss, using various bit rates for the UDP traffic. Using the same tool for measuring jitter under load, bandwidth, and packet loss was a deliberate decision to expose a multi-dimensional view of all KPIs within the same flow.

V. PHASE 1 RESULTS: LABORATORY EVALUATION
We begin the journey toward the field trials of both use cases by performing a laboratory assessment of the feasibility and challenges expected in the field. The evaluation is carried out with two different 5G network setups. The first is provided by Altice and features the same type of equipment deployed in the field. The second uses the carriergraded 5GAIner network with indoor equipment, unlike the field equipment that is certified for outdoor use and has more radios for MIMO (indoor has 4T4R vs. outdoor has 64T64R). The laboratory test results will be assessed in light of the KPIs defined in section III-B. When applicable, the relevant KPI is shown as a red dashed line in the plots. The testing methodology is explained in the previous subsection (section IV-C).

A. USE CASE 1
Use Case 1 (UC1) requires a 5G network capable of URLLC for the railroad crossing signaling. URLLC implies complying with stricter latency, jitter, and packet loss KPIs. Table 4 summarizes the KPI values that must be met for URLLC under the UC1 scenario. Therefore, we must evaluate if our networks can deliver low latency and ultra-reliability requirements. The equipment that connects to the networks and performs the vertical functionality already has reliability certifications. This subsection will present and discuss the network evaluation conducted within the ITAV laboratory using the 5GAIner network.
Starting the latency evaluation, Figure 10 shows the measured RTT histogram within laboratory conditions. The red dashed line shows the maximum RTT acceptable under the defined KPIs. We have verified that the 5GAIner network is perfectly capable of delivering the UC1 requirements. The average RTT was within 7.963 ± 2.209 ms, well below the 20 ms that was required. The maximum measured RTT was 17.4 ms, well below the target KPI. The one-way latency should be more representative of UC1, and we estimate it to be within 3.983 ± 1.106 ms.
Packet loss is critical for evaluating reliability. Figure 11a shows the measured packet loss over a 24h evaluation period. The red dashed line shows the maximum packet loss acceptable under the defined KPIs. We have verified that the 5GAIner network complied with the 0.05% packet loss KPI. Going further into the packet loss, Figure 11b shows the Cumulative Distribution Function (CDF) of the packet loss tests under two scenarios: without load in the network and with the network loaded via a second CPE performing bandwidth exhaustion (using iperf). We have determined that the 0.05% packet loss KPI is complied with even when there is load in the network.

B. USE CASE 2
Use Case 2 (UC2) targets a 5G network capable of eMBB and adequate reliability to transmit a live video stream of the   railway crossing. Unlike typical consumer scenarios, in this use case, the eMBB profile must ensure sufficient uplink bandwidth to transmit the live feed from the video camera connected via 5G.
We obtained promising results from the tests carried out in the laboratory at ITAV using Altice's 5G SA network. Starting with the packet loss, Figure 12 shows that the Altice laboratory network can deliver an acceptable packet loss KPI up to the 32 Mbit/s mark. The red dashed line shows the maximum packet loss acceptable under the defined KPIs.
We have measured the latency within the Altice-based 5G-VINNI laboratory network to be within 39.187±6.670 ms (RTT). The measured minimum RTT was 23.6 ms, and the maximum RTT was 60.7 ms. We estimate the one-way latency is within 19.594 ± 3.335 ms. A better understanding of latency distribution over time can be seen in the histogram in Figure 13a. Figure 14a shows the measured jitter, and we have verified that the Altice laboratory network can comply with all network KPIs up to 32 Mbit/s traffic. The red dashed line shows the maximum jitter acceptable under the defined KPIs.   Moving over to the 5GAIner 5G network in the laboratory environment. Starting with the packet loss, Figure 12b shows that the 5GAIner laboratory network can deliver the packet loss KPI up to 64 Mbit/s (i.e., the available uplink). The red dashed line shows the maximum packet loss acceptable under the defined KPIs.
We have measured the 5GAIner laboratory network latency to be within 27.693 ± 2.768 ms (RTT). The measured minimum RTT was 23.6 ms, and the maximum RTT was 35.5 ms. We estimate the one-way latency is within 13.847 ± 1.384 ms. A better understanding of the latency distribution over time can be seen in the histogram in Figure 13b. Figure 14b shows the measured jitter, and we have verified that the 5GAIner laboratory network can comply with all network KPIs up to 64 Mbit/s traffic. The red dashed line shows the maximum jitter acceptable under the defined KPIs.

VI. PHASE 2 RESULTS: FIELD TRIALS
After conducting the laboratory tests assessing each network's feasibility to support the pilots, we moved toward the field trials. Similarly to the laboratory results of the previous phase, the field trial results were measured in two 5G networks and will be assessed in light of the KPIs defined in section III-B. The testing methodology is explained in section IV-C.
The first immediate constraint when going outside the lab is understanding how the coverage affects our network KPIs. Out of the two networks being considered, only the 5GAIner network had sufficient coverage to perform a coverage test outside of the designated trials area (i.e., the following  subsections have the results with both networks within the designated trials area). Figure 15 shows how the RTT varies when traveling through the pilot's neighboring area. We have determined that the 5GAIner network can comply with the 20 ms latency KPI even when traveling outside the pilot's designated area.
After the laboratory tests, the equipment installation in the Aveiro Harbor began, and we carried out tests using Altice's 5G SA network. A better understanding of the latency distribution over time can be seen in the histogram in Figure 17a and summarized in Table 6. We have also measured bandwidth, packet loss, and jitter using a UDP stress test with iperf. Figure 16a shows the packet loss results up to the point of non-compliance, shown as a red dashed line. The tests stop once the network cannot deliver the required KPIs, measured at 8 Mbit/s in our trials. Figure 18a shows the measured jitter, and we have verified that the 5G-VINNI network can comply with all network KPIs of up to 8 Mbit/s traffic. The red dashed line shows the maximum jitter acceptable under the defined KPIs.
Performing the same tests in the 5GAIner SA network yielded better results. The network can comply with all network KPIs up to the 64 Mbit/s traffic threshold (i.e., the available uplink), as shown in Figure 16b. The red dashed line shows the maximum packet loss acceptable under the defined   KPIs. A better understanding of the latency distribution over time can be seen in the histogram in figure 17b and summarized in Table 6. Figure 18b shows the measured jitter, and we have verified that the 5GAIner network is a betterperforming solution. The red dashed line shows the maximum jitter acceptable under the defined KPIs.

VII. DISCUSSION AND CRUCIAL FINDINGS
While still in the laboratory and the very early stages of experiment design, we have identified that the 5G uplink limitations could play a crucial role in vertical use cases. Verticals may produce large amounts of information and expect an eMBB profile to deliver enough uplink for their needs. Notably, the second use case has a video source sending the feed via 5G directly to the consumer device (i.e., the terminal in the train). Because both ends use a 5G access network, the uplink for the video feed becomes the bottleneck. At the time of our evaluation, there were some restrictions on available RAN configuration, so a 4-to-1 slot assignment was used. That means we could only have one uplink slot for each four downlink slots in RAN -a typical arrangement even in production networks. Meeting the vertical's demands may require a higher ratio of uplink to downlink transmission slots in the RAN than we can currently configure. The issue may be alleviated using technologies such as CoMP [43] that may improve network performance, especially the data rate at the cell-edge. CoMP was developed in 4G, but Huawei (i.e., the equipment supplier and partner of the 5GAIner network) has also implemented it in 5G. Since industrial use cases will be spread across cell coverage, many use cases may be placed in coverage locations that do not have the best network performance. So with CoMP it is possible to improve the performance of the 5G network for these use cases. As seen from the industrial use case requirements, verticals require networks with uplink throughput greater than or equal to downlink throughput. 5G currently does not support it, so 5G NR needs to assign a higher amount of OFDM symbols to uplink than the downlink so that the uplink can have equal or greater throughput than the downlink. Considering that the use cases presented use device-to-device communication, one way to improve communication by reducing latency and providing direct communication is using 3GPP sidelink [44]. Sidelink was introduced in Rel-16 and is expected to provide direct communication between UEs so that the transmission between devices does not need to go through a UPF and pass through RAN twice.
We will now discuss the results presented in the previous sections, critically comparing the tests made in the laboratory against the pilots made in the field trials and highlighting the crucial findings.

A. PHASE I: LABORATORY
The laboratory tests in Phase I were crucial to understanding the challenges and tuning the networks for our use cases. For instance, Altice's 5G-VINNI deployment was iteratively improved with the collaboration from the different suppliers during Phase I until it finally entered compliance. The iterations allowed significantly reduced jitter when under load and also showed some improvements in packet loss. However, the latency did not improve significantly across those iterations. The final results for Altice's 5G-VINNI (as presented in section V) showed that the network was suitable for UC2 and could potentially support UC1. 5GAIner, with its carried-grade components, showed even better performance than Altice's 5G-VINNI deployment. 5GAIner complied with all identified KPIs (see section III-B) for both use cases. Furthermore, we have experimentally shown that the network could support the UC1 requirements in a 24h evaluation test.
In sum, after iterative improvement, we had two 5G networks that could deliver sufficient performance to support the desired field trials.

B. PHASE II: FIELD TRIALS
Driven by the successes in the laboratory tests, the Phase II results allow us to critically assess how the laboratory results correlate to the actual values from the railroad level VOLUME 11, 2023 crossing. Altice's 5G-VINNI deployment delivered very similar latency values in the field to the ones measured in the laboratory. However, these values were still less performant than the 5GAIner network. We could observe that the 5G-VINNI field network had a significant drop in throughput for the locations used during the field trial. Packet loss and jitter under load increased. As a result, the 5G-VINNI network could not fulfill the packet loss requirements of UC2. Similarly, the network could not comply with the requirements for UC1. These results show that field trials with the exact same equipment used in a laboratory may yield significantly different results (i.e., not compliant with the set KPIs). These findings track with our expectations and knowledge that distance, equipment type (i.e., indoor or outdoor), and RAN tuning play crucial roles in 5G performance. The Altice's 5G-VINNI deployment used the same indoor radios in the field.
In turn, 5GAIner had carrier-graded outdoor equipment for the field. Despite that, the field tests still show an expected decrease in performance compared to the laboratory. This performance decrease is likely due to the higher distance between the communication elements (i.e., hundreds versus tenths of meters). As a result, even though UC2's requirements can be fulfilled up to 64 Mbit/s, the network no longer could comply with the packet loss requirements at 80Mbit/s (it succeeded in the laboratory). Furthermore, the field results show that using the 5GAIner network for URLLC is possible but not guaranteed. There must be extensive validation testing in that location for continued compliance with the KPIs. Currently, the networks used for this evaluation do not have the available spectrum to deal with field noise, interference, or (moving) obstacles to the extent necessary before making any ultra-reliability claim.

VIII. CONCLUSION
The deployment of 5G brings forth a plethora of new system capabilities that allow usage scenarios to observe performance increases and create new connectivity possibilities. For the first time, new industrial sectors are looking at mobile communications as a potential key asset for information interexchange, breaking barriers between technical domains. 5G was developed considering this, broadening the deployment capability of communications technology, thus contributing to cost optimization of the players involved.
However, such changes cannot be put into practice overnight. Different domains have different languages, certification procedures, requirements, and expertise. There needs to be a progression from both ends, allowing them to meet in the middle and generate the necessary adaptations to allow a full-fledged integration and capitalization of the new technological capabilities. This paper contributed by exposing 5G deployments to specific scenarios of the transportation sector and gauging their direct deployment capability as a replacement to existing dedicated wire-based communications that need construction work and different certification processes. The preliminary results showcased that base configured 5G elements can provide good performance for the specific key performance indicators of the selected scenarios but do not yet fully fulfill them.
Thus, further enhancements are expected from both ends, with the mobile network side making available fully operating slice-based network operations beyond just enhanced mobile broadband. Additionally, considering the safe and isolated transmission of critical data (i.e., the one exchanged between train detectors in railroads and the level crossing controllers), it is important to assess their operational capability in scenarios with a higher density of devices (and users) exist.

4G
Fourth OFELIA, and 5GROWTH, where he also played key roles from proposal elaboration to task and workpackage co-leading. He is currently a WP Leader with the National 5G Mobilizer Project. Parallel to his 13 years of experience on mobility management research, he has been more recently developing work on the areas of the 5G, network function virtualization, software defined networking and information centric networking, deploying new visions and enhancements of such concepts over wireless networks, in national and international research projects. He is the Vice-Chair for the IEEE ComSoc PT Chapter.
JOSÉ QUEVEDO (Member, IEEE) received the Ph.D. degree in telecommunications from the MAP-tele Doctoral Programme in Telecommunications, in 2020. His early research activities were focused on information-centric networking (ICN) approaches for supporting the Internet of Things (IoT) scenarios. This work conducted in the Telecommunications and Networking-Av Group (TN-Av), Instituto de Telecomunicações (IT-Av), and the University of Aveiro (UA), Portugal, has been disseminated in book chapters, conference and journal papers, and contributions to open-source software. He has been involved in the different stages of several research projects (e.g., H2020 5Growth, H2020 5GASP). He has been further involved in the academy by working as an Invited Adjunct Professor at the University of Aveiro-Águeda School of Technology and Management (ESTGA). He is currently the Executive Manager of the 5GAIner Laboratory and a Senior Researcher with IT-Av working in areas related to networking protocols, network programmability, and 5G and beyond systems. CARLOS MARQUES received the Integrated M.Sc. degree in electronics and telecommunications engineering from the Department of Electronics, Telecommunications and Informatics, University of Aveiro, in July 2010. From 2010 to 2013, he has joined the Institute of Telecommunications, University of Aveiro Campus, as a Researcher. He has been involved in some national projects, such as Panorama and others in collaboration at PT Inovação, and European ones, such as Euro-NF. In 2013, he has joined Altice Labs (PT Inovação), Aveiro. He has been working on 3GPP, NFV, 4G, 5G, MEC, and cloud networking. He has been involved in some European projects, such as MCM, SUPERFLUIDITY, 5GTANGO, 5G-VINNI, 5Growth, RESISTO, CityCatalist, 5G-EPICENTRE, and FLOYD or DISCRETION. His research interests include related with orchestration platforms, VMs, containers, cloud networking, and 3GPP.
ÁLVARO GOMES received the degree in electronics and telecommunications from the University of Aveiro and joined Altice Labs, in 1984. He is currently working with the Altice Laboratories, Innovation and Technology Strategy Department, as responsible for research projects and consultant in wireless communications. His research interests include wireless communications and systems (3G, 4G, and 5G), namely: radio interface, radio resource management and optimization, radio network planning, the IoT, and self-organizing networks. He participated in international collaborative projects in the framework of European research and development programs such as IST, FP7, and H2020, as well as in national collaborative projects (P2020). He is the coauthor of two patents: Managing link layer resources for media independent handover and Context aware MBMS coverage control and has published technical papers in the most relevant magazines and conferences. VOLUME 11, 2023