A MapReduce Approach for Traffic Matrix Estimation in SDN

A traffic matrix (TM) is a source of critical traffic throughput information for traffic engineering activities and network management tasks such as traffic prediction, capacity planning, network provisioning, and anomaly detection. However, estimating TM poses several challenges for network engineers. One of the challenges is that traffic data statistics are constantly changing, and their aggregation for real-time monitoring becomes a difficult task. This paper presents a near real-time TM estimation approach for OpenFlow (OF) networks. It makes use of Big Data techniques based on MapReduce operations to tackle the aggregation problem. The proposed method uses traffic data statistics collected from OF switches through an SDN controller as input and aggregates these data in a Big Data streaming processing environment. This paper explores the benefits of the distributed MapReduce computing model to provide an estimate of the TM for all origin-destination (OD) pairs of hosts in the network in two ways: 1) the accumulated throughput and 2) the throughput between two sequential TM estimates. This procedure enables network engineers to monitor the behavior and evolution of the throughput on each OD pair in the network and on each link in the path between each OD pair. The generated TM is persisted in a NoSQL database and can be made available for a variety of network traffic monitoring applications. The results of the simulations show the potential of the proposed MapReduce approach for TM estimation.


I. INTRODUCTION
A traffic matrix (TM) provides the traffic throughput between any OD pairs in a network over a specific time interval [1], [2]. This information is essential for several activities in traffic engineering [42] (TE) such as switch load balancing, traffic analysis, and fault tolerance analysis. For example, if a link fails, a new route between the OD pair affected by this failure must be selected among the available routes that connect these nodes. To choose the best route, the volume of traffic in each possible route must be available for an optimal decision.
Due to a large number of combinations of OD pairs present in large networks, accurate measurement of the TM becomes a hard task. One of the significant challenges is the number of links between any OD pairs. As the number of links increases, providing an accurate measurement of the TM becomes even harder. The traffic on each link in the path must be retrieved The associate editor coordinating the review of this manuscript and approving it for publication was Rashid Mehmood .
to aggregate the traffic data statistics between nodes A and B at a specific time. Collecting traffic information on all links at the same time is extremely difficult because of the lack of measurement infrastructure [1], [4]. For IP networks, some previous works have inferred the TM indirectly by observing SNMP link loads [5], [6]; another approach estimated TM using sample flow statistics with Cisco NetFlow [7], [8], and Zhao et al. used a combination of both [1].
One of the features introduced by the OF protocol is the use of counters [9], [10]. Counters are maintained for a variety of objects in an OF switch. The introduction of counters enables a more direct approach to collect traffic data statistics. There are basically two approaches to collecting counter values: (1) indirectly using the SDN controller and (2) directly sending specific messages to the switches. The more common approach is to use an SDN controller, which usually, in addition to collecting counter values, provides a wide range of configuration and service request APIs. Using the appropriate APIs, an application can retrieve counter values of any switch connected to the SDN controller and estimate the TM. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ However, even with the introduction of counters by the OF protocol, two challenges for TM estimation still remain: i) which approach to use to collect the traffic data statistics, namely, direct measurement or inference techniques, and ii) how to aggregate the traffic data statistics to produce the estimated TM. Direct measurement can lead to small errors in the estimated TM [7] because this technique aims to collect traffic data statistics for all devices and all links between OD pairs in the network; however, as previously stated, this technique requires a measurement infrastructure. Inference techniques collect traffic samples and use specific methods to estimate the TM based on these samples. Both techniques provide a set of values that can be used to estimate the TM.
A common problem faced by previous TM estimation methods is related to aggregating the collected traffic data statistics [4], [28], [29], [37]. For real-time measurement, depending on the amount of collected data, calculating aggregated link, switch, and route traffic throughput may be time-consuming, making methods unable to meet real-time constraints. The aggregation process starts by storing the collected data, which arrive in a streaming fashion while previous data are still being processed. The stored data are then grouped by a pre-determined key as they arrive in random order. The desired outputs, such as link traffic throughput, are computed according to the grouped keys. Each one of the previous aggregation process steps must be addressed by any TM estimation approach.
To solve the issue of aggregating the collected traffic data statistics in near real-time, this paper proposes an approach to TM estimation that uses a Big Data programming model called MapReduce [11]- [14], [19]. MapReduce is a highly scalable programming paradigm capable of processing massive volumes of data through parallel execution. Even though this approach is considered generic enough to work with inference techniques, the proposed solution is focussed on direct measurements. The counter values for the switch ports collected from the SDN [16], [17] controller are processed by the MapReduce approach to generate the estimated TM for all the OD pairs in the network every t seconds. The challenge posed by direct measurement is the number of counter values to be processed. The counter values are initially mapped to key-value pairs, according to the OD pairs of hosts, to be later reduced to generate the TM. MapReduce not only provides straightforward primitives to generate the key-value pairs but also offers a highly scalable processing environment capable of reducing the grouped keys in near real-time. These MapReduce features are crucial for the proposed approach.
The contribution of this paper is as follows: (1) The proposed solution for the TM estimation problem uses Big Data processing techniques and tools to address the issue of aggregating traffic data statistics, which to the best of the authors' knowledge, is the first attempt to use this approach for SDN. In addition, the current research work defines the persistence of the estimated TM, which lays the foundation for further traffic analysis using processes similar to Big Data analytics. (2) The proposed approach can produce the estimated (a) accumulated TM between all OD pairs in the network in near real-time. It also provides and persists the traffic throughput in each active link in the network.
(3) In addition to the traditional TM, the proposed approach also produces what is called here the (b) actual TM between all OD pairs. The actual TM consists of the difference between the previous and the current calculated throughput for each OD pair [15], revealing the momentaneous traffic change. The power of Big Data processing tools permitted our approach to generate both TMs: (a) accumulated, and (b) actual, in the same processing window. The generated TMs enable network managers to keep track of traffic evolution: i) between any OD pair, and ii) on each link that composes the path between any OD pair, in near real-time. The recurrent estimation of both TMs under a restricted time requirement is simply achievable by using a powerful programming model like MapReduce. This becomes more evident if a more detailed view of network resources usage is needed.
The motivation of the proposed approach is that direct measurement can be feasible if appropriate infrastructure is provided, not only for collecting the counters but also for aggregating the collected traffic data statistics.
The remainder of this paper is structured as follows: Section II presents related work concerning TM estimation. Section III describes the proposed MapReduce approach. Section IV shows the results of the simulations performed in this study. Finally, section V concludes this paper.

II. RELATED WORK
The TM plays an essential role in the field of network management and traffic engineering and has been extensively investigated over the years.
For traditional IP networks, several different approaches were used to estimate TM. Soule et al. [20] created a state space model to capture temporal and spatial correlations between pair of hosts and the applied Kalman Filters to estimate and predict TMs. Xie et al. [21] proposed a Sequential Tensor Completion algorithm (STC) to recover missing Internet traffic data and also prosed a Reshape-Align scheme [22] to reshape inconsistent traffic matrices and align these matrices to form a tensor. Polverini et al. [23] proposed mrT, which is a MapReduce-based framework for estimating the TM of an Internet Service Provider (ISP) network.
The OF specification defines a number of flow counters that can be used to increase the accuracy of the TM estimate. Tootoonchian et al. [4] presented OpenTM, a traffic matrix estimator for OF networks. OpenTM was implemented as an application for the SDN controller to monitor active flows by polling the switches periodically. Tootoonchian et al. [4] also explored various algorithms for choosing which switch to query. The difference between OpenTM and the present work is that in this study, ports are monitored instead of flows. Moreover, the previous paper created flows between specific pairs of hosts and provided statistics for these flows.
The MapReduce approach proposed here can provide the statistics between any pair of hosts, even if no explicit flow has been created between them.
Gong et al. [28] proposed two strategies to design traffic measurement rules to be installed in the OF switches flow tables. They assume that rules for routing flows in each SDN are aggregated, whenever is possible. These aggregated routing rules can be used to estimate the TM. In their work the aggregated rules are disaggregated to improve the estimation accuracy. Li et al. [29] developed a method to determine which flows are most informative to construct a measurement flow set iteratively until an accuracy requirement is satisfied or a measurement resource constraint is reached. FlowMon [30] provided a sample and fetch based mechanism to detect large flows. The sample mechanism detects suspicions large flows and the fetch mechanism install measurements rules in the OF switches. OpenSample [32] leverages sFlow packet sampling to provide near real-time measurements of both network load and individual flows.
Luong et al. [34] proposed a solution for monitoring throughput of link traffic and a new forwarding algorithm for the control plane. It is based on two modules for the SDN controller. The Throughput Monitor module focuses on the number of packets and bytes that crossed the switch. The Packet Forwarding module implements the forwarding algorithm. The statistics provided by the Throughput Monitor module are available for use by any monitoring application. Volpato et al. [35] proposed OFQuality, a module for QoS configuration based on the OVSDB protocol, which benefits from TM generation to manage QoS policies on the OF switches.
Tian et al. [36] proposed a new framework for TM estimation in an SDN-based IP network. Their work shows that if a flow's traffic can be derived from other flows in the network, adding a new entry for this flow for traffic measurement does not provide new information about the TM.
iSTAMP [37] used a well-compressed aggregated flow measurement and the K most informative flows to infer the TM. iStamp partitions the flow tables into two parts: the first for aggregate measurements, and the second for per-flow monitoring of selected flows. Using these two distinct parts, iStamp estimates the traffic matrix.
Jiang et al. [38] developed an algorithm to estimate and recover the network traffic matrix at fine time granularity from sampled traffic traces. Their algorithm is based on fractal interpolation, cubic spline interpolation, and the weighted geometric average algorithm.
Azzouni and Pujolle [39] presented NeuTM, a framework for TM prediction based on Long Short-Term Memory Recurrent Neural Networks. NeuTM trained a neural network using historical traffic data to predict the future TM.
Choudhury et al. [40] described an application of machine learning for TM prediction in SDN-Enabled IP/Optical networks. Their work used machine learning for short-and long-term prediction of all elements of the traffic matrix. The previous studies mentioned above estimated the TM based on modelling and optimization procedures. For modelling, mathematical models were used to estimate the TM based on sample mechanisms for traffic data statistics. The proposed MapReduce approach uses direct measurement and, with the help of big data tools, tries to minimize the TM estimation error. The proposed approach monitors switches and ports traffic data volume. Using this method, all traffic that crosses a link between an OD pair is used to estimate the TM.

III. MapReduce APPROACH FOR TM ESTIMATION
The proposed MapReduce approach is part of a TE method based on Big Data techniques shown in Fig. 1 [41], [42]. This TE method delivers fine-grained statistical analysis of network resources, such as the ratio of each port in the switch payload, throughput capacity used on ports, and the TM estimation. Two modules compose the TE method: The Big Data Streaming module and the Traffic Engineering module. The traffic statistics data coming from an OF switch can be collected in a streaming fashion because of the fast changes in the counters due to dynamic traffic crossing the switches. The Big Data Streaming module collects network data and forwards them to be processed according to the data purpose (Flow Data, Topology Data, Controller Data, and Traffic Data Statistics).
The Traffic Engineering module performs four TE activities as shown in Fig  The proposed MapReduce approach can produce several throughput measurements including the TM estimation method. The focus of this paper is on TM estimation, using the online traffic data statistics provided by the Big Data Streaming module, for the monitoring activity.
The proposed approach for TM estimation is described in detail in the next subsections. Subsection A provides details of how to process the traffic data statistics related to switch ports that are needed to calculate the traffic throughput in every link formed in the network. It is then followed by the definition of the data structure required to store the network elements and topology data in Subsection B. Finally, the algorithms to estimate the TM are described in Subsection C.

A. MapReduce DESIGN
The TM estimation consists of aggregating the traffic throughput of all links in the path between an OD pair of hosts. This aggregation becomes harder when the number of possible combinations of OD pairs increases due to the number of hosts in the network. This scenario poses two challenges: 1) identify the links that connect all OD pairs, and 2) aggregate the values of all links to provide individual OD throughput estimation. For a static solution where the network topology is previously known, and the links are easily identified, the aggregation process is easier to implement. The MapReduce approach proposed in this paper, on the other hand, considers that the network topology is provided by the SDN controller and, from the textual description of the network topology, derives the links connecting all OD pairs of hosts. Once the links are identified, a Map operation generates all the K , V pairs, where the K contains the identification of the link and the OD pair that this link connects, to be aggregated for each OD pair of hosts every t seconds, and V is the link calculated throughput. Hence, all values of V , with the same K , will be added to generate the estimated throughput for each OD pair of hosts.
The design described in this subsection provides the functions needed to estimate the TM. The initial requirement to estimate the TM is to collect traffic data statistics related to switch ports. As shown in Fig. 1, the collection process is carried out by the Big Data Streaming module, and this process is detailed by Queiroz et al. [41]. For the context of this paper, it is considered that the Big Data Streaming module provides the traffic data statistics as messages, which are used as input for the proposed MapReduce Approach. These messages are referred to as Raw Data because they deliver the number of bytes that crossed a switch port at a specific moment. These data are used to calculate the traffic throughput in the switch port and need to be organized to reflect links throughput (two ports of distinct switches compose a link).
The next requirement for the TM estimation is to identify the links that connect an OD pair and aggregate the traffic throughput of these links. The MapReduce approach initially calculates the throughput for each port and identifies the links formed by each pair of ports. Next, the links for each OD pair are aggregated using a unique key that represents that OD pair. The last requirement to estimate the TM is to sum all the calculated throughput with the same key to generate the estimated TM; the Reduce function performs this summation.
The MapReduce approach for the TM estimation consists of four map functions, and one reduce function as shown in Fig. 2.
The Map Raw Data function receives raw data (traffic data statistics) as messages and generates a key/value map where the timestamp of each message is the key; and the data needed to calculate the port traffic throughput are the value.
Next, the Sort By Key function sorts the keys (timestamp) generated by the Map Raw Data function in ascending order. This step is necessary to calculate the current link throughputs. The current link throughput is used to calculate the current throughput between OD pairs. The maps generated by the map functions are shuffled before they are processed by the next function. Shuffling is a default operation performed by MapReduce tools to enable distributed processing. However, the map generated by the Sort By Key function needs to be processed in the same order that it was generated, i.e., in ascending order by timestamp because the current link throughput is the difference between the previous and current throughput. To avoid the shuffle in the map produced by the Sort By Key function, the function Coalesce is executed.
The Generate Final Map function generates a map that provides the current and accumulated link throughput that is aggregated to estimate the TM. This map is the last requirement to estimate the TM.
The Reduce by Key function processes the final map to estimate and persist the TM.
The MapReduce paradigm enables the approach for the TM estimation to be focused on the solution and not on the implementation details. MapReduce abstracts the data structure to store the keys to be aggregated. Also, the ability to process the aggregation of the keys in parallel saves great effort in the design of a parallel environment to reduce the keys. These features enabled our approach to deliver not only one TM, but two TMs (accumulated and actual), which is an advance when compared with previous solutions. In addition, the approach proposed here did not select one specific OD pair to monitor, but all OD pairs in the network topology.

B. DATA STRUCTURES TO ESTIMATE TM
This subsection provides the second requirement to estimate the TM, i.e., the data structures to store hosts, switches, switch port, and links data that will be used to generate the keys. All these data need to be identified and structured to reflect the network topology. This is necessary to identify all the links that are in the path between two OD pairs. This section introduces the basic definitions of all these data along with their structures.
The definitions are as follows: This set represents the network nodes.
This set represents all hosts in the network.
This set represents all switches in the network. Each switch s i has several ports, which can be represented as s i1 , s i2 , · · · , s il .
This set represents host/switch connections.
This set represents switch/switch connections.
• The network is represented by a graph G given by (N , U ).
This set represents the links between host and switch ports.
This set represents the links between two switch ports.
This set represents the combinations of OD pairs.
• A Path P from h i → h j is a sequence h i , s 1 , s 1 , s 2 , s 2 , s 3 , · · · , s k−1 , s k , s k , h j such that each switch in the sequence s 1 , s 2 , · · · , s k is distinct.
, · · · , (h n , h m )) | i = 1 · · · n, j = 1 · · · l}. This set represents all paths that include the switch port s ij . To generate the TM, some basic prerequisites are required: 1) The sets H (hosts), S (switches), and the graph G representing the network are given. : Because each element in HC is a binary relation between members of H , the value for r is 2.
3) The set L (links between host and switch ports and links between two switch ports) is given. The LP set provides, for each switch port, the list of OD pairs for which this port is part of the path. To build this set, the first step it to build the HC set, which contains all the possible combinations of OD pairs. For each pair in HC, the path P is retrieved. Each path between the members of an OD pair consists of three types of links: (1) a link between the origin host and the first switch, expressed as h i , s 1 , (2) links between switches, expressed as s i , s j , and (3) a link between the last switch and the destination host, expressed as s k , h j . For each type of pair, the set L is searched to find the ports used in the link. For link types (1) and (3), a match is sought for elements in L 1 (links between hosts and switch ports). Elements in L 1 are described as (h i , s j , s ja ) or (s j , h i , s ja ). When a match is found, the switch port (s ja ) is returned. For link type (2), the same procedure is followed, with elements in L 2 (links between two switch ports) searched for matching. In this case, when a match (s i , s j ) is found, both ports(s ia , s jb ) are returned. For each returned port, a pair ( port , (O, D)) is added or updated in LP.

C. ALGORITHMS TO ESTIMATE TM
The LP set is the primary data structure in the proposed MapReduce approach because it provides, for each link, all OD pairs that the link connects.
The data provided by the Big Data Streaming module (Fig. 1) represent the raw data on the switch ports. This section presents the generation of the LP set and the procedure that combines the collected raw data with the LP set elements to generate the final map to estimate the TM.
The Map Raw Data function (Fig. 2) receives traffic data statistics as messages; these messages have the layout provided in Fig. 3. The timestamp field is in the Unix format, where the last three digits refer to the millisecond of the data collection time. The approach used here discards the last three digits because it provides the throughput in seconds. This operation changes the timpestamp 1522598210107 to 1522598210 and makes it possible to group statistics collected in the same second. This grouping operation enables the TM estimation to take place at every t seconds.
The map generated by the Map Raw Data is sorted by the Sort By Key function. The sorted key is composed of the timestamp and switch port fields. This sort function enables the estimation of the current throughput, which is the difference between the previous and current calculated throughput.
The Generate Final Map function provides two algorithms that are needed to generate the final map. The goal of these algorithms is to generate a map (final map) holding the throughput information in every link based on the topology and the raw data collected from the network. The Generate LP set algorithm (Algorithm 1) generates the LP set, and the Generate Key, Value pairs algorithm (Algorithm 2), for every port statistics message, calculates the throughput of the link that include this port and generates the Key, Value for all the OD pairs that include the port. The following sections describe these algorithms. VOLUME

1) GENERATE LP SET ALGORITHM
Algorithm 1 shows the steps to generate the LP set. The LP set includes, for each switch port, the list of OD pairs for which this port is part of the path.
Topology information is collected from the OpenDaylight SDN controller to build the sets H (hosts), S (switches), U (host/switch connections and switch/switch connections), and L (links between hosts and switch ports and links between two switch ports). OpenDaylight's response comes in a JSON format and is converted to an internal CSV format. Fig. 4 shows the topology used in the experiments. The elements of L are directly related to the elements of U because they provide a critical piece of information, the port number in each switch used in a connection. Therefore, for each element of U 1 (h i , s j ), the switch port (s ja ) is added to form the elements of L 1 . The host port is not used in L 1 because it is assumed that only one network interface is available in the host.
The elements of L 2 are generated in the same way; for each element of U 2 (s i , s j ), the ports in the connection (s ia , s jb ) are then identified.
The graph G is built using GraphStream [43], which is a dynamic graph library in Java. The approach used here extracts the nodes (hosts (H ), switches(S), and links(U )) and feeds the graph with these values.
Lines 8-13 from Algorithm 1 generate the HC (OD pairs) set. For each OD pair (h i , h j ) in HC, the path connecting the two hosts is returned by the function getPath(h i , h j ) (Lines 15-16). In the topology presented in Fig. 4, if an attempt is made to find the paths between any two hosts, loops can be found in the network graph. To overcome this issue, the topology was divided into two parts: the first with S1 as the root, and the second with S2 as the root. With this approach, the solution presented here provides the TM between all possible combinations of OD pairs traversing S1 or S2.
As previously mentioned, the approach used here divides the network topology into two parts to avoid loops in the network graph. Loops in the network are not the only problem faced when traversing a network graph. Another problem is to find the shortest path between an OD pair. Even though the Dijkstra algorithm solves this problem, it is still possible to find more than one shortest path between an OD pair. In this case, one of the shortest paths can be selected for monitoring. The MapReduce approach proposed here enables the monitoring of more than one shortest path. The solution is to add a different identifier to the generated OD pair definition in LP. For instance, in a network with two shortest paths from host Ha to host Hb, where the first shortest path includes switch port SAp1 and the second shortest path includes switch port SBp2, the entries in LP for SAp1 and SBp2 can differentiate the two shortest paths by creating OD pair definitions. The presence of the strings PATH1 and PATH2 differentiates the OD pair definitions for the two shortest paths.

2) GENERATE key , value PAIRS ALGORITHM
The goal of this algorithm is to generate the final map. This map contains the traffic throughput for every link in the network. The link is not part of the key; only the OD pairs that are connected through this link compose the key. The map consists of OD pairs as the key and the traffic throughput of the link as the value.
The pseudo-code provided in Algorithm 2 describes the Generate key, value pairs algorithm. The generated key, value pairs compose the final map, which is used by the Reduce By Key function (Fig. 2) to estimate the TM.
For each port traffic data statistics message (Fig. 3), the message of the port pair with the same timestamp is retrieved (Line 8). A port and its pair are an element of the set L. If the port is an element of L 1 , the port and its pair have the same value because in L 1 the link is formed between a host port and a switch port, but the host port is not monitored. For L 2 elements, (s i , s ia , s j , s jb ), if the fields second and nanosecond are exactly the same for (s ia , s jb ), the value of bytes received (BR) for s ia should be the same as the value of bytes transmitted (BT) for s jb , and the other way around. However, collecting statistics for (s ia , s jb ) at the same second  and nanosecond is an unrealistic scenario. Hence, one of the messages has the most up-to-date statistics for the link.
To use the most up-to-date statistics, the message with the larger value of (BR + BT) represents the throughput of the link (Line 10). Fig. 5 provides an example. The ''S1:eth1'' and ''S3:eth3'' ports form a link, and the statistics of the ''S3:eth3'' port are used as the throughput of the link in that timestamp.
For each message (Fig. 3), two calculations are performed: 1) Current throughput in every link. The current throughput is calculated according to Equation 2 [15]: where T i (t) is the current throughput of link i at instant t, c i (t) is the statistics value (BR + BT) at instant t, and T is the polling interval (Line 11). Lines 9 and 13 respectively retrieve and update c i (t − T ) for each link. 2) Accumulated throughput in every link. The accumulated throughput is calculated according to Equation 3 (Line 12): where T i (t) is the accumulated throughput of link i at instant t and S is the number of seconds for which the link is active. The final step in Algorithm 2 is map generation. The LP set is queried to provide the paths that include the selected port (Line 14). For every returned path, a key, value pair is generated (Lines 15-19). Table 1 provides an example of the generated key, value pairs for the S3-eth3 port. The key is composed of the path and timestamp, and the value contains the calculated current (C v , Equation 2) and accumulated (A v , Equation 3) throughputs.
For all port traffic data statistics in the same timestamp, the same type of key, value pairs as those generated for the S3-eth3 port and shown in Table 1 are generated.
The Reduce By Key function aggregates values with the same key. Table 2 shows the reduce operation applied to   some of the key, value pairs presented in Table 1. The TM consists of records composed of the fields provided by Table 2 (Key, Current, Accumulated). Figure 6 shows the TM computation procedure. Messages are processed in message sets, and each message set is composed of all the messages that arrive in the Queue Messages module during one MapReduce execution. During the processing of Batch 1, the messages that compose Batch 2 arrive in the Queue Messages module. At the end of Batch 1 processing, the computed TM is sent to storage. During the processing of Batch 2, the messages that compose Batch 3 are still arriving. This is a continuous process. The MapReduce module is scheduled to execute every t seconds.

IV. EXPERIMENTS AND RESULTS
In this section, we present the real-time measurements provided by the MapReduce approach. The topology used in our experiments is the one presented in Fig. 4.
The experiment is robust enough to challenge the MapReduce paradigm in two different ways: i) the number of keys created for each OD pair of hosts, because of the number of links presented in the topology. This procedure also demands the Map operation to sort all the created keys to set up the TM with the current throughput. ii) network basic traffic statistics arrive in a streaming fashion, so, at every t seconds the two TMs (accumulated and actual) are generated by Reducing the created keys.
Four machines were used to run the experiments. They are described in Table 3.  Each big data tool was configured in a cluster formed by four Linux containers running Linux Ubuntu Server 16.04. A feature called Multipath was configured in the servers enabling the containers to have multiple I/O paths to the storage, allowing parallel access to the four configured partitions.
Mininet was used to create the network topology and run the experiments. Using Iperf3, we created TCP flows according to the scheduling diagram shown in Fig. 7. This diagram shows the generated traffic between pairs of hosts and the respectively configured bandwidth. We also added some flow table entries to change the default route configuration provided by OpenDaylight. These flow entries define layer 3 routing, as shown in Table 4.
The collected traffic data statistics processed by Spark streaming were persisted in Elasticsearch every 3 seconds. Even though all calculated throughput were provided in realtime, the graphics were generated off-line. The graphics provided in this section show the throughput for selected links and OD pairs, but the MapReduce approach provides the throughput for all links and OD pairs in our topology. The scheduling diagram shown in Fig. 7 displays the bandwidth selected to every generated traffic. The bandwidth value is expressed in Mbits per second.
To validate the TM, packets statistics were captured on the switches ports using tcpdump. Using tshark and a python script, the throughput at every 3 seconds for each link was calculated, and then plotted along with the calculated throughput provided by the MapReduce. This plot is intended to show that the MapReduce calculated throughputs follow the same pattern as the one calculated using tshark. With tshark we calculated the statistics at every 3 seconds and the counters values provided by OF standard includes seconds and nanoseconds, which did not allow us to match the values exactly at the nanosecond level. Thus, it is not expected the values calculated using tshark and the ones provided by MapReduce to be the same.
The MapReduce approach generates the current and accumulated throughput between all OD pairs in the topology. As these values are calculated using the individual values on each link, we provide the results not only for OD pairs but also for individual links.

A. CURRENT LINK THROUGHPUT
The current throughput of a link is calculated according to Equation 2. Fig. 8 shows the current estimated throughput at every t − T time interval in the links formed by S2:eth4 and S6:eth4, S1:eth1 and S3:eth3, and S2:eth3 and S5:eth4. The link S2:eth4/S6:eth4 is used by the traffic from h1 to h7 and from h4 to h8. S1:eth1/S3:eth3 carries the traffic from h1 to h5 (at 0 and 25s) and from h2 to h8. The traffic from h2 to h6 at instant 10s drives the throughput in S2:eth3/S5:eth4.

B. LINKS ACCUMULATED THROUGHPUT
The accumulated throughput in a link is calculated according to equation 3. Fig. 9 shows the estimated throughput in the links formed by S2:eth4 and S6:eth4, S1:eth1 and S3:eth3, S2:eth3 and S5:eth4, and S1:eth2 and S4:eth3. The throughput in link S2:eth4/S6:eth4 increases during the traffic from h1 to h7, experiences a small decrease, and then  stops decreasing because of the traffic between h4 to h8. The throughput behaviour observed in link S1:eth1/S3:eth3 is driven by the traffic between h1 and h5 and between h2 and h8. The traffic from h2 to h6 is the only instance that uses the link S2:eth3/S5:eth4. Link S1:eth2/S4:eth3 is used by the traffic from h3 to h7.

C. HOST-TO-HOST CURRENT THROUGHPUT
As shown in Table 2, the current throughput of an OD pair is an aggregation of the values of individual links between the hosts. Fig. 10 shows the estimated current throughput between h3 and h8 (root S2). The first four top graphics show the throughput on each link that connects the hosts, and the graphic on the bottom shows the aggregation of the individual links. The links between h3 and h8 are: S4:eth1, S2:eth2/S4:eth4, S2:eth4/S6:eth4, and S6:eth2. The only traffic flows in S4:eth1 is that from h3 to h7, which starts at 25s. The S2:eth2/S4:eth4 link is used by traffic from h4 to h8, which starts at instant 50s. The traffic from h1 to h7 at instant 10s and from h4 to h8 at instant 50s drives the S2:eth4/S6:eth4 link throughput. The S6:eth2 link carries traffic from h2 to h8 at instant 45s and from h4 to h8 at instant 50s.

D. HOST-TO-HOST ACCUMULATED THROUGHPUT
The OD pair accumulated throughput is also an aggregation of the values of individuals links between the hosts. The MapReduce approach provides an estimate of the throughput VOLUME 8, 2020  between all possible combinations of OD pairs traversing S1 or S2 according to Table 5. Fig. 11 shows the estimated throughput between the following pairs of hosts: h1-h5 (root S1), h3-h8 (root S2), and h2-h4 (root S2). Table 5 provides the rules for verifying which ports are used by each OD pair. For instance, the following links connect h3 to h8 using S2 as a root: S4:eth1, S4:eth4/S2:eth2, s2:eth4/S6:eth4, and S6:eth2. The OD pair h1-h5 has two specific traffic states defined at instants 0 and 25s. No traffic occurred between h3 and h8, meaning that the links that connect them were used to estimate throughput. In this case, the following traffic flows were used in throughput estimation: h1-h7, h3-h7, and h4-h8. The same situation applied to the OD pair h2-h4. In this case, the traffic between h1-h7, h2-h6, and h4-h8 generated the throughput for this OD pair.

E. PERFORMANCE EVALUATION
The proposed MapReduce approach was executed inside an Apache Spark Streaming environment. A performance evaluation of the execution of the proposed approach based on several metrics produced by the Spark Streaming monitoring system is now presented. Fig. 12 shows the record input rate statistics. The average value of 7.62 records per second is shown on the left side of the figure. The graph in the centre shows the evolution of the input rate over the simulation period. The first observed spike from left to right is related to the time that Spark takes to set up the environment, which causes an accumulation of records. Subsequent spikes are produced by the records arriving every 3 seconds. Spark streaming processes the input    records in batches, and the histogram on the right side shows the distribution of batches/records per second.
Scheduling delay is the time between the instant that a collection of jobs for a batch was submitted and the instant that the first job was started. The scheduling delay for the present simulation is shown in Fig. 13. According to the graph in the centre, the first value for the scheduling delay was over 2 seconds because of the environment set-up, but the following batches had values near zero, making the overall average 173 ms. The histogram graph on the right shows that most of the batches had nearly zero scheduling delay time. Fig. 14 shows the processing time for each batch of data. The average processing time in the present simulation was 2.5 seconds. According to the graph in the centre and the histogram in Fig. 14, most batches finished their execution in less than the defined time to process each batch (3.5 seconds, as shown by the dotted line labelled stable). The reason why this time is not the same for all batches is that the record input rate (Fig. 12) is not the same. The record input rate is related to the Apache Kafka record processing time and affects the time that all MapReduce functions (Fig. 2) take to run in a single batch. The time needed to persist each set of calculated statistics is already included in the batch processing time.
According to Spark Streaming documentation, the total delay parameter shows whether or not the system is stable. The system is considered stable if it can keep up with its record input rate. If the delay is maintained to be comparable to the batch size, the system is considered stable, which is true in the approach proposed here as shown in Fig. 15.
The results provided by the metrics used to evaluate the Spark Streaming performance show a stable running environment capable of seamlessly estimating the TM. Any online traffic monitoring system requires this stability because of the relevance of the TM for activities such as traffic prediction, capacity planning, and anomaly detection. Also, the results endorse the use of Big Data techniques to aggregate network traffic data statistics.

V. CONCLUSION
Estimating the TM in real time is a challenging task. Two common approaches to tackle this problem are direct measurement and inference techniques. This paper proposes a solution to this problem based on the MapReduce Big Data programming paradigm. The importance of the proposed approach is the use of direct measurement, which poses several challenges. One of the main challenges is to aggregate the large amount of collected data between all OD pairs and to produce the TM in real-time. The proposed approach uses the data distribution and parallel processing features of MapReduce to estimate not only one, but two TMs (accumulated and actual) in near real-time. This result shows the potential of Big Data processing techniques to estimate the TM using direct measurement of network traffic.
The MapReduce paradigm speeds up TM generation because it can aggregate keys in parallel. In the next steps of this research, the authors will examine the scalability of the proposed approach as the numbers of hosts and switches in the network topology are increased because increasing the number of hosts and switches will make collecting traffic data statistics even harder. In addition, in the context of real-time TM estimation by direct measurement, the use of different tools of Big Data streaming processing tools that implement the MapReduce paradigm will be evaluated.