An Overview of Current IP Network Emulators for the Validation of Railways Wireless Communications

,

systems comprise subcomponents of different suppliers. This effort is carried out with the aim also to reduce costs and time compared to on-site testing as well as increasing the number and type of tests that can be done. Focusing on the communication technologies, testing all candidate technologies for an application makes the on-site testing unfeasible in terms of cost and time. In order to overcome this issue, the number of tools and strategies developed for testing the communication in the laboratory is increasing. These different strategies for testing in the laboratory are simulation, Hardware (HW) in the loop, and emulation (explained in section III).
It is important to find a laboratory validation tool in order to search for limitations and improvements of a new technology, approaching the closest to the on-site testing in a controlled laboratory environment. Nevertheless, the performance of a communication network does not depend exclusively on the technology but also on the different impairments that the environment causes to, particularly, in wireless networks, for example, the multipath effect [6].
Especially railways do not present a clean environment. In fact, railways present the same problems as other areas (e.g., multipath effect) but additionally, some effects arise just only in this area, e.g., the effect in the communications of transient electromagnetic (EM) disturbances produced by the sliding contact between catenary and pantograph [7], [8].
In this paper, different railway applications and services that will be enabled with the new communication technologies are explained (section II) and, the different test strategies that could be applied to test them are listed (section III). Moreover, current network emulators are presented (section IV) to, later on, compare their main features and characteristics (section V). Finally, conclusions are drawn (section VI).

II. RAILWAY APPLICATIONS AND SERVICES
Railway applications and services based on communication technologies are mainly focused on the safety-critical operation or passenger services related to the comfort. Due to these different goals, the applications making use of communication technologies have different set of requirements. The different railway applications could be classified into two types: critical related to safety and non-critical related to non-safety and passengers comfort.
On the one hand, the ones caring about the safety-critical operations are more restrictive than the ones related to the passengers comfort dealing with high availability, low delay, and low packet loss [9]. This type of applications includes signaling systems such as ERTMS (European Rail Traffic Management System) or CBTC (Communications-Based Train Control). Any of these systems could send messages containing, e.g., information for an emergency brake, being a critical message. Safety-critical applications use communications to interact between the train and the control center at trackside. Taking into account that this kind of operation could put passenger's life at risk, communication technologies have to be intensively tested to characterize the performance in different scenarios. These scenarios could be related to different communication technologies, channel access modes, cyber-security attacks, or different environment and impairments conditions, which lead to a huge number of combinations.
On the other hand, non-critical applications as telemaintenance or passenger connectivity do not request such critical restrictions; if the data arrives with a higher delay time than the critical ones (e.g., 4s) it does not have a critical impact.
In the ideal case, the communication between train and ground is continuously working and never disrupted. However, in the real world, communication can be affected by the environment where the train is operating, namely a forest, urban canyon, or conflict areas in terms of perturbations. All these areas have a negative effect on the communication channel, reducing the performance of the communication due to multipath effects or interferences in the signal transmitted having as result a degradation or even loss of the received information. The high-level communication architecture between the train and the ground with any application (APP) is shown in Fig. 1. A communication channel is set-up between the train and the base-station transceiver to enable the bi-directional information flow. The communication channel that allows sending data between the train and the ground can be implemented by different technologies. For example, in mainline tracks, the current ERTMS, consists of ETCS (European Train Control System), focused on the signaling protocols, and its communication technology based on GSM-R [9]. Although GSM-R is the current communication channel for ERTMS, it is becoming obsolete due to its limitations [10], [11]: 1) GSM-R cannot provide advanced services and is not able to be adapted to new requirements, such as critical video application, which needs more data rate than the GSM-R one. The maximum transmission rate of GSM-R per connection is 9.6 kbps, which is sufficient only for applications with low data rate demands such as ETCS. As well, message delay is in the range of 400 ms, which is too high to support any real-time application and emergency communication [10]. VOLUME 8, 2020 2) Interferences: the interference between GSM-R and GSM public network increases because both railways and public operators aim at having coverage along the rail tracks [10]. Theoretically, such interference can be avoided if public operators do not use frequency bands adjacent to those of GSM-R for the areas close to rail tracks; however, this is not well implemented in practice. As well, other EM interferences disturb GSM-R, such as the sliding contact between the catenary and the pantograph [12]. 3) Capacity limitation (spectrum): GSM-R has no sufficient resources for the next-generation railway system, where each train will need to establish a continuous data connection with a Radio Block Center (RBC), and each RBC connection needs to constantly occupy one-time slot [10], [11]. Because of these limitations, new technologies for railways are being researched in order to achieve the capabilities required by new services. Due to the current global communication architectures, these new communications technologies for railways are moving towards Internet Protocol (IP) based solutions, such as LTE-R. This technology is being specially researched in railways due to the performance and level of maturity of LTE, such as [13] stated. Moreover, the first LTE-R network of China is scheduled for 2020 [10]. Furthermore, some European projects are researching which solutions along different possible technologies will be the op-timal ones such as the Shift2Rail project.

III. TESTING STRATEGIES AND EMULATORS
This section introduces the current communication technologies testing strategies and network emulator characteristics.

A. TESTING STRATEGIES
When a new technology or application is developed, it has to be validated before putting it into service. In fact, the testing strategy for the system under test is crucial to have a correct validation. This can be done by different strategies, so it is important to clearly understand the difference between them in order to know the different capabilities that each of them offers. These strategies are implemented in different environments starting from the laboratory by means of simulation, HW in the loop (HIL) and emulation, and finally, when the laboratory tests have been successfully passed, on-site testing. Fig. 2 shows a complete proposal of validation taken into account the different strategies named above. Knowing the differences between all these strategies is key to understand what each can offer. It should be pointed out that in the literature [14]- [17], there is no agreement with regards to the simulation and emulation definitions. Therefore, the following definitions of simulation, HIL, and emulation are used in this paper: 1) Firstly, simulation is defined as a software-based technique that is based in an analytical model, which represents the key characteristics, behaviors, and functions of the real system describing them using mathematical tools for a system virtualization in a controlled environment. Simulation usually does not require specialized hardware equipment, which can significantly reduce the final cost of this experimental technique. The disadvantage of this approach is the fact that the real environment needs to be generalized to describe the real system. 2) Then, the hardware in the loop (HIL) is a technique where real hardware is present in the simulation loop, being the testing and the evaluation of the system carried out in real-time [17]. HIL is most often used in the development and testing of embedded systems when those systems cannot be tested easily, thoroughly, and repeatable in their operational environments [18]. This testing strategy is more focused on the subsystem level than in a whole system under test. The system under test is thought as the final assembled product.

3) The emulation is an experimental technique, which
replicates the same inputs and outputs as the real system with the same performance, e.g., being able to test in real-time. The main goal is to test a whole system. 4) Finally, on-site testing is defined as testing the whole system in the real world with a real environment. The on-site testing in railways is difficult and expensive. Some reasons are the following ones: the non-full availability of the infrastructure and the dynamic change of the environment. Therefore, it can be said that the emulation and the simulation are the options to check the performance of a whole system under test. However, HW in the loop is not the most suitable strategy for testing a whole system under test but for testing a specific equipment meaning subsystem testing.
Nevertheless, these laboratory testing strategies have some advantages and disadvantages versus on-site testing, as it is shown in Table 1. The main advantages of laboratory strategies are the reduction of costs and time and the option of repeating the test the times that the user desires. However, for an effective testing an important disadvantage is the need to overcome the challenge of having a realistic environment prior to deployment in order to reproduce coherent network conditions. For this purpose, the emulation can be considered because it is close to the real world, and it replicates the real-time performance of the emulated system. However, it is hard to obtain in the emulation the same results as the on-site testing due to the fact that the real environment conditions produce unpredictable impairments. In fact, one of the main challenges is to accurately identify which are the best and worst use cases conditions in which networks and devices will have to work in. Moreover, it is difficult to shift every possible scenario from on-site testing to the laboratory.
Comparing emulation and simulation, it can be easily seen that the emulator has to be in real-time, because it employed the final equipment, while simulator can be faster, slower or in real-time because it employs simulation models. This implies that the emulator is expected to have the same behavior (or close to) of a real system, same inputs, and outputs as the real system, allowing a closer real performance evaluation.
With regards to on-site testing, due to the costs, availability, and not repeatability among others, it is clear that the fact of shifting tests from on-site testing to the laboratory is necessary. In this field of moving on-site testing to the laboratory there are a number of European projects such as EATS [20] included in FP7 EU research funding program; and VITE [21], X2RAIL-1 [22] and X2RAIL-3 included in Shift2Rail initiative [4]. The main objective is to reduce on-site tests for signaling systems (in these cases, focused on ETCS railway application), leading to reduce overall testing costs under the zero on-site testing concept. In fact, the key objective of zero on-site testing is to perform functional and non-functional tests (component test, integration test, and system test) in laboratory, instead of testing on-site, in order to save time and costs without compromising safety [23]. As well, the laboratory allows a controlled environment that on-site does not; the on-site carries limited scope and non-controlled environment (hard or even impossible to control the nature).
Therefore, bringing to the laboratory (emulation or simulation) at least a representative number of use cases from the real world is required. Nevertheless, it should be taken into account that some of the use cases could only be tested on real site testing; they are not possible to be shifted to laboratory such as brake testing.
The complete and most desirable testing process would be to pass through every laboratory testing strategy allowing to characterize the infrastructure performance before taking the deployment decision, and once the deployment is completed on-site test could be carried out. However, a trade-off is necessary due to the cost restrictions of covering all the testing strategies proposed. Therefore, considering that the emulator is the testing strategy closest to the real world, it is considered the best solution to shift the on-site testing to the laboratory.

B. NETWORK EMULATORS CHARACTERISTICS
As it is stated in section II, different elements along the track cause different effects on the communication channel. Some of these effects that disturb the communication channel at RF level are multipath or EM effects. These disturbances caused in real world have to be taken into account when testing. Communication Network Emulation tools are capable of inserting different impairments to the channel with the aim of reproducing real-world condition effects, approaching the best to reality.
As mentioned in section II, the current trend is to move forward to IP based connection schemes. When employing an IP emulator, these RF disturbances are present but indirectly; although the disturbances affect the channel directly at RF level, they do it indirectly at IP level. Hence, the RF interferences are present and possible to be translated into IP network impairments if the research of how they affect to IP level is developed. A list of IP level emulators is listed in section IV.
With regards to the IP level the network emulators, the following IP impairments are defined: 1) Packet delay: is the time required for each bit of the packet to traverse the network or a segment of the network, independent of the packet size [24]. 2) Bandwidth: in terms of data network, bandwidth quantifies the data rate at which a network link or a network path can transfer; the amount of data a link or network path can deliver per unit of time [25]. 3) Packet corruption: deletion, insertion, modification, re-ordering in a packet in terms of bits. 4) Packet loss: the failure of a packet to traverse the network to its destination [24], [26]; it is measured as a percentage of lost packets with respect to sent packets. It occurs when one or more packets transmitted over an IP network fail to arrive at their destination. Packet loss is typically caused by what is generally referred to as network congestion but also because of distance or poor line quality. Excessive packet loss is perceived as disconnections (broken or missing communication). Thanks to the possibility of modifying these impairments, it is possible to change the conditions of the environment, so it enables to test both the train to ground signaling application and the communication technology that is used to transmit the different messages. Therefore, in order to select the most suitable network emulator, it is needed to have a look into the characteristics of the different current applications that are being developed for the sector (explained in section II), in this case, railway area.
Some criteria can be applied in order to choose the best suitable network emulator. In fact, the three features shown in Fig. 3 are the main criteria from the functionality point of view. These functional features are explained below: 1) Performance: the degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage [27]. 2) Precision: refers to the closeness of two or more measurements to each other. 3) Repeatability: the user can carry out the same test under the same conditions the times that the user wants. If an emulator has a high performance, high precision, and high repeatability, it is for sure an emulator closer to the real world. Nevertheless, these features are not the only ones when choosing a suitable emulator; other important non-functional criteria are cost and time constraints (not represented in Fig. 3).
Different network emulators, which are currently in the market, are explained and compared in the following sections, section IV and section V, respectively.

IV. CURRENT IP NETWORK EMULATORS TESTING
In this section, a number of IP-based network emulators available currently in the market are listed and briefly described. First, the difference between HW and SW emulators is explained according to Fig. 4, where the definition of terms of high and low depends on the explained concept. For example, low physical ports means that the HW emulator has a fixed number of ports, and it is not possible to increase them.
Usually, HW based emulators are standalone devices, implemented on dedicated HW platforms, equipped with an operating system and software suitably optimized and customized. Thanks to these characteristics, these solutions usually grant excellent performance and accuracy certified by the manufacturer. On the other hand, they usually are very expensive [28].
However, despite the apparent limits of SW network emulators, they are widely used in many contexts and especially for research applications. The main reason is their being open source and, consequently, they can be easily modified by the researcher for their goal [28]; the SW emulator could be adapted to the user's requirements. As well, due to the high portability of the SW emulator, as they are computer programs, it can be installed in any device: they just need a host hardware platform to be executed.
In the rest of the terms, the concept of ''high'' carries positive meaning; therefore, being ''low '' negative meaning.
The features of both types of emulators are not only those explained in section III (performance, repeatability, and precision) but also portability and limitation of ports: 1) Portability: the ease with which a system or component can be transferred from one hardware or software environment to another [29]. 2) Limitation of ports: HW based emulators are limited physically with a fixed number of ports. However, the SW ones are device-independent, so they can be deployed in any device with the possibility to add more ports. 3) Performance and precision: HW emulators, respecting SW ones, are dedicated devices to the emulation task, and its operating system may be optimized for this purpose offering higher execution speed and higher accuracy of the effects being introduced [30]. However, SW based ones have to share the processor with other tasks.
Another classification of the emulators refers to proprietary or commercial network emulators and open source network emulators. Both terms have to described as they are two relevant points in the emulators. The proprietary or commercial network emulators are the ones which its software is owned by the individual or the organization that developed it. In contrast, the open-source emulators refer to the ones that have been developed and tested through open collaboration meaning anyone with the required academic knowledge can access the source code, modify it, and distribute his own version of the updated code.
Firstly, the proprietary emulators are listed as follows: 1) Spirent Network Emulator [31]: it provides industryleading flexibility in building and modelling these complex real-life systems enabling the user to emulate networks and the real-world conditions under which applications and platforms need to perform. 2) PacketStorm IP Network Emulators [32], [33]: it reproduces the unfavorable conditions of IP Networks and WANs in a controllable and repeatable laboratory setting. The emulator recreates the dynamic behavior of the Internet such that any network model can be reproduced, including those models that change with traffic, time, or the behavior of another traffic flow.
3) IXIA [34]: it allows users to accurately emulate the real network conditions that occur over live production LAN/WAN networks. By emulating realistic and worst-case network conditions in the lab, users can validate and test performance of new hardware, protocols, and applications to prevent failures in production networks. 4) NetDisturb [35]: it allows disturbing flows over IP networks, helping to study the behavior of applications, devices, or services in a disturbed network environment. 5) Linktropy 8510 [36], [37]: it emulates terrestrial, wireless, satellite, internet, and other wide area networks to test applications under a spectrum of real-world conditions.
Then, the open-source emulators are the following ones: 6) WANem [38], [39]: it is a tool, which brings the Internet into the user's development/test/laboratory environment. It emulates internet under conditions within the user's control so that the user can check application performance and availability. 7) Dummynet [40]: it is a live network emulation tool, initially designed for testing networking protocols, and since then used for a variety of applications, including bandwidth management. It enforces queue and bandwidth limitations, delays, packet losses, and multipath effects. 8) NIST Net [41]: it is a general-purpose tool for emulating performance dynamics in IP networks. 9) NetEm [42], [43]: it provides network emulation functionality for testing protocols by emulating the properties of wide-area networks.

V. COMPARISON BETWEEN THE DIFFERENT EMULATORS
Each of the emulators listed in section IV has different aspects; thus, a comparison is needed in order to see which one is the best option according to the user requirements.
In this section, a comparison is shown focusing on the main IP impairments (explained in section III) that almost all the network emulators can modify. However, modifying these impairments does not just affect the emulated communication channel, but also the performance of the different applications that go through it, e.g., video or voice, can be worsened.
In this section, a comparison by IP impairments of many emulators is shown; nevertheless, throughout the literature more specific comparison between emulators involving a specific aspect exists (usually just two emulators at the same time and just one IP impairment) [44], [45].
In Table 2 and 3, it is shown the different characteristics of each network emulator (from section IV) according to the information provided by its corresponding vendor.
In Tables 2 and 3, there are some relevant differences with regards to the specific parameters between the analyzed emulators: 1) The most obvious one is that some emulators are commercial and the other ones are open source. This usually is one important feature of the emulator depending on the user's budget and support. In Table 2  and III there are six proprietary emulators and four  open-source emulators. 2) The network emulator with higher accuracy, in terms of the configurable network impairments, would allow more precision in the results. Therefore, having high repeatability due to the same inputs would carry the same outputs. For example, in this comparison, PacketStorm6xg is the more accurate emulator in terms of delay due to the accurate resolution that is capable of providing to the user. 3) Another difference, which requires a more in-depth analysis, is the selection of HW or SW based emulators (See Fig. 4). The HW ones, e.g., Spirent emulator, have a given hardware device with some physical characteristics as up to 16 ports; if the user needs more ports, it is not possible to add more. However, the SW emulator can be adapted to any desired device, e.g. NetEm emulator. 4) The quality of the available documentation of the emulator could also be a drawback, i.e., some emulators as NetEm whose source code is constantly modified, and there is no well-documented description of its functionalities. 5) Another important topic is whether the emulator is a generic emulator that could be applied for a specific sector, or it is specific for a given sector. In the case of the emulators compared, all of them are generic.
The rest of the features, in general, differ ones from others and depend on the user; it could be more or less interesting as the ease of the network emulator usage. For example, the applications environment of the network emulator. The emulators are used as well to test the performance of different types of applications. Depending on the application that the user wants to test, the choice of the emulator would be different. Another example, real-time audio information is delay-sensitive, and excessive packet loss decline in voice quality [46]. Therefore the IP impairments, which the user has to focus on, are delay (and jitter) and excessive packet loss, which affect the quality and produce distortion. Another example of applications is the real-time video, which normally has more data to transmit. The IP impairments that degrade the different types of videos are the same as in audio applications but including another aspect: the bit data rate normally is higher than in audio, therefore, the bandwidth [47]. However, other types of applications such as exchange and retrieved information (e.g., FTP, mails) are focused on loss and bit error rate [48], [49].
These examples would need a different choice of network emulators. Consequently, the real-time audio and video applications would be suitable to be tested with a network VOLUME 8, 2020  emulator which delay is accurate such as PacketStorm HurricaneV or PacketStorm6xg (which has high BW to test video applications). However, for data transfer applications, a network emulator having the option of packet corruption and more accuracy in loss would be more suitable such as Linktropy 8510 or NetEm. Throughout the literature, it can be found that many experiences have been carried out with different applications and the network emulators exposed in section V as [50]- [55] state in order to test the performance of the target applications.
The network emulators allow the application to test them in different network environments allowing the research of the network performance impact of the application service. However, the common characteristic is the fact that all of them can repeat tests.
From the railway point of view, it can be stated that none of the emulators analyzed includes railway aspects, i.e., the impairments that the user could introduce in the emulator are in a static position not in movement as a train does. Therefore, due to the different environments present on the track, dynamic configurable impairments are needed as input in the emulator in order to represent the effects of these areas, such as tunnels or urban areas, matching with the train position. In this manner, the effects in the on-site testing could be shifted to the laboratory replaying the real scenario and allowing to test different applications. Therefore, it has to be taken into account that this need is missing and has to be developed.

VI. CONCLUSIONS
The future railways' applications, being more restrictive the safety-critical ones, request demanding requirements for the communication technologies, such as large bandwidth, low delay, and high availability. Therefore, in order to test the functions of the different applications based on communications, the performance of the communication technology has to be validated prior to putting it into service.
Currently, extensive on-site tests are considered for the validation, however, due to the high cost, time demand, and lack of repeatability, on-site tests could be considered unfeasible for the whole validation process. In order to overcome this drawback, this paper proposes a testing strategy where, firstly, laboratory tests are performed, and then on-site tests getting as result that the network emulator is the best option at laboratory level to approaches the closest to the real world and allowing to reduce the on-site tests.
As it is shown in this paper, many network emulators are currently available in the market, being able to modify different parameters of the communication channel. Choosing the suitable network emulator depends on the user requirements, e.g., what application has to be performed and/or the budget. By means of the analysis and comparison of the different characteristics of the emulator shown in this paper it can be stated that there is currently not a universal emulator consequently, the user should be able to choose the most suitable network emulator according to his/her requirements.
With regard to the railways case, it has been shown that no one of them is specific for the railway environment. The need of a railway network emulator is given by the necessity of testing different environments that the train could be passing through (tunnels, urban areas, suburban areas. . . ), each one is, indirectly (due to e.g., multipath effect) modifying the characteristics of the communication channel. Because of this, a network emulator being able to take into account the location of the train will improve the test and validation process. In this manner, it is possible to include the negative effects of the perturbations listed beforehand related to the corresponding railway environment, e.g., the tunnel in a specific position reducing the coverage. Nowadays, the need of a specific emulator for railways still exists.