Software-Defined Networking meets Software-Defined Radio in Mobile Ad hoc Networks: State of the Art and Future Directions

The aims of this survey article are to elaborate on cross-layer optimization, Software-Defined Networking (SDN) and Software-Defined Radio (SDR) as separate domains of wireless network design for which a unified view has not been adequately considered to date and present lessons learned, with a view towards the challenges associated with SDN-SDR interaction that would facilitate benefits in cross-layer optimization of mobile ad hoc networks (MANETs). We focus on MANETs because (i) they are still at the forefront of technology, and in some scenarios they are the only meaningful option for establishing communication; (ii) they expose the full potential and benefits of coexistence and interaction of SDN and SDR, in terms of optimizing key performance indicators. While SDN and SDR are mature technologies, their interaction and joint consideration have been largely overlooked. Current SDN approaches do not span the physical (PHY) and medium-access control (MAC) layers, but they rather concentrate on network-level routing and traffic flow optimization. As a result, PHY- andMAC-layer related parameters which notoriously affect key network performance metrics remain static or at best are adapted based on some heuristic or local approaches. On the other hand, the reach of SDR architectures is restricted to the PHY and MAC layers. We discuss the state of the art of cross-layer optimization, SDN and SDR, and current challenges associated with coexistence and interaction of SDN and SDR. Such an interaction would extend the span of SDN to PHY and MAC layers and lead to realizations of centralized approaches across all layers so as to control and optimize parameters, towards global network objectives. It would also create a bridge between centralized network control that is inherent in SDN and the distributed nature of MANETs, with the add-on features of flexible and fast PHY and MAC layer adaptation offered by SDR, for solid, autonomous and ultimately better network control implementations that span all layers, towards realizing and implementing the holy grail of real cross-layer optimization.


I. INTRODUCTION
T HE last three decades have seen significant progress on wireless multi-hop networks and wireless ad hoc networking. Wireless networks initially inherited the traditional layered architecture of wire-line networks, where each communication layer is treated as a separate entity with its own adaptable parameters and constraints. While independent consideration of different layers led to simplified protocol design, it often proved to be insufficient and suboptimal when dealing with wireless systems.
In the late 1990s, cross-layer design and optimization of wireless networks emerged as a major research thread, and multi-layer optimizations across two or more layers have been proposed, analyzed and implemented, for mobile ad hoc networks (MANETs) and for wireless cellular networks. The basic idea of cross-layer design is that Open Systems Interconnection (OSI) layers (physical, access, network, transport, application) and the network control mechanisms they support are considered jointly and not separately. For example, MAC-layer protocols exploit the additional flexibility pro-vided by PHY adaptation, while PHY actions need to obtain a more "network-wide" view by taking into account the impact of parameter adaptation of one node on other links and nodes. Cross-layer design extends to the network, transport, and application layers and provides the required adaptivity across all layers so that the best possible QoS is obtained. Again, the central idea is to view each global QoS metric as a function of multiple control variables at different layers and try to optimize this function jointly over all these control variables.
Software-Defined Radio (SDR) as a technology appeared in the early 1990s and has matured a lot in the last two decades. SDR enables flexible adaptation of PHY and MAC layer mechanisms on the fly. The vision in SDR is to replace all (or as many as possible of) the traditional hardware modules of transceivers with appropriate software modules that receive as input a digitized version of the down-converted received signal. Many platforms handle multiple PHY layers on a single hardware platform by means of programmable hardware controlled by software. A variety of SDR platforms and commercial products are available, with different PHY and MAC layer reconfiguration capabilities as well as different conceptual and implementation aspects around the central SDR vision [1].
In the last decade, Software-Defined Networking (SDN) has been a game-changer in network-layer routing mechanisms, since it enables centralized orchestration of network nodes through a central SDN controller. When the network is large, a hierarchical or distributed SDN architecture with several controllers where each controller is responsible for a part of the network) can address network scalability issues. SDN separates the control and data plane functionalities and places the former in dedicated network nodes, the controllers, which hold the full network information state. Therefore, distributed MANET routing protocols are replaced by centralized flow optimization algorithms on graphs. It is through such algorithms that the controllers determine and distribute the forwarding tables (known as flow tables) to other network nodes, commonly referred to as SDN switches, which have the simpler task of forwarding data packets according to their flow tables entries. Hence, SDN has brought forth and amplified the potential of centralized routing optimization algorithms which was very hard to realize before.
Historically, SDN was first applied to wire-line core networks, and later expanded to the wireless last mile of infrastructure-based networks such as Long Term Evolution (LTE) cellular and WiFi. However, its use on mobile ad hoc networks has been limited, primarily due to: (i) the high link volatility and network state variation rate, (ii) the fact that controllers may face single-point-of-failure issues, and (iii) the general hardness of reconciling the centralized nature of SDN with the hitherto distributed nature of mobile ad hoc networks.
The centralized routing imposed by SDN falls squarely within the centralized QoS optimization advocated by crosslayer design and optimization, which is discussed above. A connecting bridge, namely the SDR technology paradigm should bring MAC and PHY layers forward and in close interaction with SDN, and allow versatile adaptation of PHY and MAC parameters.

A. SDN AND SDR INTERACTION IN MANETS
While SDN and SDR have matured quite a lot as separate technologies and communication paradigms, their interaction and joint consideration have been largely overlooked. Specifically, current SDN approaches do not span the PHY and MAC layers; they rather concentrate on the network flow level and traffic routing optimization. As a result, PHY and MAC related parameters -such as transmission rate, transmission power, coding rate, channel (frequency / time slot) allocation (in case of channelized systems), channel access (in case of random access), scheduling, and queue management -which notoriously affect key performance metrics of the network remain static or at best adapt based on some heuristic or local approaches.
An interaction of SDN and SDR would extend the reachability of SDN down to MAC and PHY layers, thus leading to realization of a centralized approach spanning all layers, in which parameters from all layers would be jointly controlled and optimized towards the global network objectives and metrics. Chief among such metrics are end-to-end throughput, average end-to-end packet delay, and energy consumption to support a given number of sessions. Figure 1 depicts a standard SDN architecture applied to a MANET with SDR-based mobile nodes. The standard SDN architecture consists of an Application, a Control, and a Data (i.e., Infrastructure) Plane. Network control and management applications run in the Application Plane. The development of these applications is facilitated by a standardized Application Programming Interface (API) known as the Northbound Interface (NBI). Representative examples of such applications are shown in Figure 1. One or more SDN controllers at the Control Plane (CP) communicate with the SDN switches at the Data Plane (DP) to collect network status information and send to them commands, such as updates to flow table entries, in order to implement the policies and decisions taken at the Application Plane. Controllers communicate with switches through the Southbound Interface (SBI) and among them via the East-Westbound Interface. Note that the SDN controllers and switches are software modules that do not necessarily run on different physical nodes. Indeed, a physical node may host both a controller and a switch module. Similarly, the interconnections between a controller and a switch denote logical links that can be implemented at the physical level by a direct or multi-hop wireless connection, or even by operating-system (OS) level system calls or message passing in case both modules are running on the same hardware. An (optional) interconnection of the MANET to the infrastructure-based network through one or more fixed base station (BS) / gateway(s) is also accommodated in this architecture.

B. ARTICLE CONTRIBUTION AND ROADMAP
The aim of this article is to survey cross-layer optimization, SDN and SDR as separate pieces in the puzzle of wireless network design, for which a unified view has not been considered to date, and to argue that their joint consideration can bring benefits to MANETs. A first reason for the focus on MANETs is that these are still at the forefront of technology and arise in many mainstream applications, while in some scenarios they are the only meaningful option for establishing communication. On the one hand, MANETs encompass several traditional application scenarios that are still very active, such as army tactical scenarios, disaster rescue ad hoc networks, and sensor data collection. On the other hand, they are the cornerstone of several modern application scenarios such as smartphone device-to-device networks, networks of Unmanned Air (or Surface) Vehicles (UAVs, USVs), and vehicular ad hoc networks (VANETs), to name a few.
A second compelling reason is that MANETs can expose the full potential and benefits of coexistence and interaction of SDN and SDR, in terms of optimizing key performance indicators, through global and network-wide cross-layer design and optimization. Finally, there exist important challenges to SDN-SDR coexistence on the implementation front as well, which can be better exposed in a mobile ad hoc network terrain.
It should be stressed that the intention of this article is not to propose a new specific plausible architecture for joint SDN-SDR operation in MANETs. Rather, our goals are to elaborate on cross-layer optimization, SDN, and SDR as separate domains of wireless networking and present lessons learned, with a view towards the challenges associated with SDN-SDR interaction that would facilitate cross-layer optimization. To the best of our knowledge, such a survey does not exist in the literature.
The structure of the article is as follows. In section II we present an overview of the theoretical foundations of crosslayer design and optimization, which form the substrate for SDN-SDR cooperation. The overview is centered around the Backpressure (BP) algorithm which lays a solid framework for cross-layer network control in MANETs. We discuss centralized and distributed approaches and comment on lessons learned that pave the way to SDN. In section III, we provide a survey of SDN approaches, categorized based on the way SDN controllers and switches exchange control-plane messages. Categories include in-band and out-of-band control messaging as well as hybrid and few-hop control messaging approaches. In section IV, we present the background, drivers and technology behind the efforts for SDN-SDR coexistence in wireless networks. Current SDR standardisation efforts are presented while the few existing works on SDN and SDR coexistence are discussed and some thoughts on the challenges of SDN-SDR interaction are presented. Finally, section V concludes the paper and presents various directions for future research.

II. CROSS-LAYER DESIGN AND OPTIMIZATION
In this section, we provide an overview of the main conceptual approaches for cross-layer wireless network design and optimization. Cross-layer design and optimization of wireless networks have been extensively studied during the last three decades. Cross-layer interaction in wireless networks is motivated by a number of reasons, such as the dynamic nature of wireless networks, the variability and lower capacity of wireless links, and the fact that the wireless medium is shared among several users. While medium sharing is also adopted in some cases of wire-line protocols such as Carrier-Sense Multiple Access with Collision Detection (CSMA/CD), where users use the channel for transmission one at a time otherwise there is collision, the interference situation in wireless networks is much more complicated as simultaneous transmissions in adjacent links are typically not possible, unless Multiple-Input-Multiple-Output (MIMO) techniques or different frequency channels / spreading codes are used.
Co-channel interference between users that reuse the limited spectrum, and the resulting impact of local adaptation actions on overall network performance are the prime reasons for encountering layer interactions in wireless systems. Thus, for example, physical layer (PHY) parameters, such as transmit power or modulation scheme have considerable impact on multiple access of users in a common channel since they affect interference levels as well as the amount of interference that can be sustained in the channel. Adaptation of such parameters affects not only the quality of service (QoS) at the PHY layer, e.g., the bit-error-rate (BER) or the signal-to-interference-and-noise ratio (SINR), but also the perceived QoS at higher layers, e.g., achievable end-to-end session transmission rate.
Over the years, several classes of works on cross-layer design and optimization have been proposed, with each class having numerous representative papers. Each of these classes of works is based on concepts that address interactions between two layers, and primary the interaction between PHY and the medium access control (MAC) layer. For example, sophisticated signal processing at the receiver combined with random access schemes allows the physical layer to receive multiple packets at the same time, and it addresses the interactions between the physical and medium access layers. Another class of works considers the interaction of PHY and Data-Link control (DLC) layer through the impact of modulation level, coding rate, and transmit power control on Automatic Repeat reQuest (ARQ) schemes. Yet another class of works makes use of throughput information from the MAC layer in making the routing decisions at the network layer. Finally, another aspect of cross-layer interaction involves the design of mechanisms to make transport-layer protocols work in the presence of wireless links. For example, the explicit congestion notification (ECN) at the transport layer can explicitly tell the transmitter if there is congestion in the network to enable it to differentiate between errors on the wireless link and network congestion.
In this section, we focus our attention on the celebrated Backpressure (BP) algorithm because it provides a solid and systematic framework for realizing cross-layer optimization in MANETs, where PHY and MAC layer control policies affect routing decisions. The superiority of BP over other cross-layer protocols lies in that BP presents a principled approach, driven by mathematical optimization that addresses the three-layer interaction, namely PHY, MAC and network layers. Furthermore, it brings into the picture the two basic sources of variation in wireless networks, namely dynamic wireless link conditions and traffic variation. The latter is taken into account through traffic queue dynamics at nodes which affect the routing and scheduling decisions. Finally, the optimization approach of BP can be extended to the transport layer through a holistic network utility maximization (NUM) perspective. First, we present the concept and some variants and extensions of BP. Next, we present an overview of relevant distributed and centralized methods. The works discussed in this section are shown in Table 1.

A. THE BACKPRESSURE ALGORITHM FOR CROSS-LAYER NETWORK OPTIMIZATION
Chronologically, one of the first and most important systematic approaches towards a cross-layer design vision in multi-hop wireless networks has been the Backpressure (BP) algorithm [2], [3]. The BP algorithm tackles precisely the inherent dynamic nature of wireless networks due to stochastic traffic, link volatility, fading, interference or node mobility. It addresses the fundamental problem of identifying a network control policy that is throughput optimal (namely, it guarantees bounded node packet queue lengths), whenever the input traffic arrival rates are in the network stability region. The network stability region is the set of input traffic arrival rates for which there exists a network control policy such that packet queues are stabilized. The BP algorithm has been shown to be throughput optimal.
The BP algorithm operates on a time-slotted system as follows: • At each time slot, for each link, it computes the maximum differential backlog over all flows. • Then, it observes the instantaneous network topology state and it searches over the control space to find the control policy that maximizes the sum of products of maximum differential backlogs and link transmission rates over all links in the network. • If the control space is the set of all feasible scheduling policies, then the subset of links that are selected for activation in the second step are activated. If the control space includes other control variables, these are also applied. Packet transmission for each link follows, according to the prescribed transmission (queue service) rates from the second step. At its basic version, the BP algorithm finds the subset of links to activate such that the sum of products of differential backlogs and PHY transmission rates is maximized. Thus, BP prioritizes transmission at each time slot for links that have (i) maximum differential backlog, that is, maximum difference in the queue lengths between the transmitting and receiving node of the link; (ii) high potential transmission rate, and this prioritization offers a means to perform traffic routing.
The control space may involve other mechanisms at the access and PHY layers besides link activation scheduling which affect link transmission rate, such as channel selection, transmit power control, or PHY-layer modulation and coding rate control. This interdependence among different layer mechanisms gives rise to a cross-layer approach. If the control space of these mechanisms is discrete, identifying the optimal control policy at the second step of the algorithm above is a combinatorial optimization problem which is usually hard to solve. Several approaches have focused on proposing centralized heuristic solutions for such problems e.g., joint channel allocation, modulation level, and transmit power control so as to maximize total PHY transmission rate per time slot [4], [5], or joint time slot allocation, link activation scheduling, and transmit power control to minimize power consumption [6], [7].
The original BP algorithm does not use pre-specified paths. Namely, different packets of the same flow can follow different routes to the destination. Furthermore, BP does not imply that a packet will be routed through links to continuously move towards the destination, and this may cause delays in the network. Several enhancements to the basic BP algorithm have been proposed, aiming at the reduction of end-to-end delay, which is a known problem of BP. An enhancement to the basic BP algorithm has been proposed in [8], with the aim to reduce delay. In that approach, link weights also depend on the difference of distances of the end-points of the link from the destination. With this enhancement, packets tend to follow shortest paths towards the destination. Finally, BP for throughput optimality is one facet of a general class of stochastic optimization approaches that aim to optimize the network for other performance metrics such as energy consumption [9]. The reader is referred to [10] for a comprehensive approach on the topic.
In subsequent works, cross-layer design also spanned the transport layer with the aim to find the optimal traffic injection rates in the network so as to maximize total network utility, subject to network flow and other resource constraints [11]. A joint optimization perspective has been proposed, grounded on convex optimization that naturally maps to several different layers of the protocol stack: congestion control at the transport layer, routing at the network layer, and link scheduling and power control at the MAC/PHY layer. Layer interaction is facilitated through the circulation of Lagrange multipliers that arise from different constraints. These Lagrange multipliers are interpreted as the scalar multiples of queue sizes of nodes. With the circulation of Lagrange multipliers, the congestion control and scheduling decisions are made independently. Based on this concept, various works have studied distributed cross-layer designs spanning also the transport layer [12], [13].
A generalization of the concept of [11] followed in [14]. The overall goal of devising optimal network control mechanisms that span the transport, network, access and PHY layer mechanisms is formulated as a global Network Utility Maximization (NUM) problem. This global problem is decomposed into a set of sub-problems, where each subproblem corresponds to a separate layer, while the interaction among layers is realized through coordinating variables, predominantly the Lagrange multipliers. Different layers iterate on different subsets of the decision variables using local information to achieve global optimality. Some concerns about cross-layer designs subsequently emerged [15], primarily about the risk of sacrificing cleanness of the architecture for performance optimization. Concerns such as that of a resulting "spaghetti" cross-layer design or the possibility of missing important interactions that cross-layer design entails came to the surface. The paper above advocated the "totality of the design", implying the need for "holistic rather than fragmented" architectures.

B. DISTRIBUTED VS. CENTRALIZED APPROACHES
The survey [16] presents a taxonomy of centralized and distributed Backpressure protocols. In centralized approaches, the routing and scheduling control is exercised by a central controller residing at a central location, server or node. The controller obtains the global network state information (i.e., the link status, queue lengths, etc.) and solves a global optimization problem, whose outputs are the control decisions at each slot. While centralized protocols in general achieve high performance in terms of throughput, the computational tractability of the global optimization problem is questioned, especially when networks grow large.
On the other hand, in distributed protocols, each network node can make localized routing and scheduling control decisions by using the network state information in its disposal. This information can either be the full global network state information (although in this case, there still exists the issue of how often this information will be updated), or just local information. However, several issues arise, such as that of having inconsistent and out-of-sync state information at different nodes, for example about queue lengths. Given the high rate of variation of queue lengths, the availability of precise queue lengths at each time slot is difficult, which in turn renders the computation of the precise queue differential backlogs at each time slot challenging as well. If (some portion of the) global state information is needed at some nodes, the issue of large communication overhead arises. Distributed versions address the scalability challenge of centralized protocols in terms of required state information and control communication overhead.
In BP, while queue differentials can be computed in a distributed fashion by communication of each node with its neighbors, the determination of the link schedule and link transmission rate requires the solution of a centralized (global) optimization problem for which information from the entire network is needed. Two interference models are commonly assumed: (i) the interfering link set model, whereby each link is associated with a subset of conflicting links called interfering link set, and the link cannot be scheduled if some other link in that set is scheduled; (ii) the node-exclusive (primary) interference model, in which the interference set of a link includes all links that have a common end-point with that link, and a node cannot transmit and receive simultaneously. For the latter model and the simple case where the control space includes only link activation scheduling, the optimal subset of links to activate emerges through a maximumweight matching problem that needs to be solved at each time slot. A matching in a graph is defined as a subset of links with no common end point, and the weight of each link is the queue differential of that link. In addition to the computational complexity of O(N 3 ) for finding the optimal solution for a network of N nodes, significant control message overhead is created for collecting the network state (i.e., topology and queue) information at the central location, and subsequently for disseminating the solution to the entire network at each time slot.
In an effort to develop distributed solutions, a subsequent line of works sought low-complexity approximate maximal matching solutions instead of maximum-weight matching ones [17], [18]. A maximal matching is a set of matching links for which the addition of another link in the matching set does not make the set a matching set any more. These papers show that such maximal-matching solutions can achieve throughput at least 1/κ and 1/2 of the maximum possible one respectively for the two interference models above, where κ is the maximum number of links that can be simultaneously scheduled within any interfering link set. Parameter κ is computed as follows. For any link , the interfering link set is the set of links S such that cannot be active if some link in is active. However, any two (or more) links in S can be scheduled together, and κ is the maximum such number of links that can be scheduled together across all sets S i.e., for all links . The work [17] also proposes low-complexity distributed algorithms that approximate the performance of maximal matching based scheduling. The throughput-optimal decentralized approach in [19] is based on a combination of a distributed matching algorithm and an algorithm that compares and merges successive matching solutions. This approach leaves open the issue of control signaling and associated overheads.
In most of the works above, the efforts to provide a distributed solution to the problem focus on decentralizing the computational problem. However, most of the approaches above still imply the existence of a control channel through which all nodes can communicate and exchange information, possibly in a multi-hop fashion. A decentralized approach to BP that tries to bypass both the control signaling dissemination and the computation of maximum-weight matching is proposed in [20]. At each time slot, each node decides randomly to transmit or not. If the node decides to transmit, it broadcasts a pilot signal to its neighbors at some reference signal strength. All nodes measure the resulting interference level at their receiver and broadcast the information to their neighbors. The resulting signal-to-interference levels for all outgoing links of a node are then found, and the prospective transmission rates on each link can then be computed using Shannon's capacity formula. Finally, each node decides to transmit with full power to the neighbor with the largest product of link queue differential and link transmission rate. In [20], it is shown that the distributed approximation above leads to queue stability for input data rate that is about half of the corresponding input data rate of the centralized BP algorithm for which there exists queue stability.
After the theory was established, few algorithms entered the implementation realm. The work [21] implements XPRESS, a throughput-optimal BP architecture for wireless multi-hop networks. A central scheduler performs BP scheduling based on the measured wireless network state, while wireless nodes periodically provide to the controller network measurements and realize the computed schedules. XPRESS integrates the transport, network and MAC layers in the implementation with a traffic congestion control scheme, a coordination mechanism between network-layer flow queues and MAC-layer link queues, and a multi-hop Time Division Multiple Access (TDMA) MAC protocol. The work [22] presents two efforts in the context of the OPNEX European project, towards implementing BP or protocols inspired by it. These are XPRESS, and another protocol termed Queue Length-aware Routing (QLR) whose main idea is that each node selects the flow and neighbor to forward packets to based on the maximum product of internal queue length for a flow, and expected packet transmission time over the link to that neighbor.
The implementation of a distributed version of BP is presented in [23]. The paper makes two major contributions: first, it uses a practical link cost metric, the ETX that are required so as to successfully transmit a packet from a sender to a receiver. This cost is incorporated into the classical BP metric of link queue differential so that priority for link transmission depends also on low expected number of transmissions over a link. Second, a last-in-first-out (LIFO) queue service is proposed instead of first-in-first-out (FIFO), which leads to reduction in packet delivery delays.

C. REMARKABLE ISSUES ABOUT CROSS-LAYER OPTIMIZATION
• The vast majority of the proposed approaches above are centralized and have been developed using advanced optimization theory, algorithm analysis and design. However, only a handful of them have entered the implementation realm. • Most efforts towards providing distributed solutions to the cross-layer optimization problem of mobile ad hoc networks primarily aim at decentralizing the computation part of the required global optimization problem through message exchange between the central location and the network nodes. Other approaches try to bypass the computational burden of the global problem and the issue of control message overhead through localized randomized heuristic approaches with significant losses in throughput performance. • A performance comparison between centralized and distributed algorithms is currently lacking. Some works provide theoretical guarantees for distributed solutions based on established results on approximating the solutions to known graph-theoretic problems. Some other works, e.g., [20], compare the centralized and distributed solutions through simulations on toy networks under certain assumptions. Furthermore, no works have provided solid comparative results between centralized and distributed methods in real implementations. • While the fields of cross-layer optimization algorithms analysis and design have advanced a lot, important practical repercussions in the implementation of crosslayer design and optimization approaches are largely overlooked. Most importantly, the majority of centralized approaches assume or imply the existence of a multi-hop control channel for control information dissemination between the central controller and network nodes in multi-hop fashion. However, the control traffic overhead and end-to-end communication delay caused by this multi-hop information exchange are commonly underestimated in the literature. • While several works in cross-layer optimization have demonstrated significant performance gains over the classical layered designs in wireless networks, there is no cross-layer standardized architecture with associated established protocols. As detailed in the sequel, we advocate that SDN offers a solid and well-established framework and basis to fill this important gap for mobile ad hoc networking since it has been successfully used in wire-line networks for over a decade. SDN can provide the bridge between centralized network control that is inherent in SDN, and the distributed nature of mobile ad hoc networks. The incorporation of SDR in the architecture can offer a means for flexible, fast and ultimately better implementations that span the access and PHY layers through agile local adaptations of lower-layer parameters at network nodes.

III. SDN FOR MOBILE AD HOC NETWORKS
Over the last decade, Software-Defined Networking has found widespread acceptance as a networking technology in wired networks (i.e., telecommunications carriers' networks, Internet Service Provider (ISP) networks, and data centers). SDN standards (such as OpenFlow), open source SDN controller and switch implementations, and major telecom vendor SDN products have all contributed to promoting SDN as a mature and well accepted technology. That said, virtual network functions (VNFs) on general purpose hardware cannot reach the performance level of specialized hardware and are still away from replacing core internet routers which can handle huge amounts of network traffic. SDN has also been applied to infrastructure-based wireless networks (cellular and WiFi) and included in the 3rd Generation Partnership Project (3GPP) and European Telecommunications Standards Institute (ETSI) cellular network standards. The introduction of SDN into MANETs, however, is a challenging problem as the concept of centralized control, which is the cornerstone of SDN, and the high volatility of MANETs are rather contradictory. Nevertheless, the topic has received a growing attention in the literature with a number of works arguing that the benefits that SDN brings to any network provide a strong motivation to overcome the challenges of applying SDN centralized control to MANETs.
Maintaining controller-switch connectivity may be challenging in cases of a rapidly changing topology due to high mobility. In such cases distributed MANET routing protocols, such as the Optimized Link State Routing (OLSR) protocol, may work better. However, in several use cases, quasi-static or low mobility scenarios can be very often encountered in practice and in that case SDN is a plausible solution. An overview of the motivation behind introducing SDN to tactical ad hoc networks, the challenges raised by the ad hoc and coalition network environment, as well as some proposed design concepts and methodologies to ad-dress these issues are discussed in [24].
In this section, we review the literature on SDN-based MANETs and describe a variety of different design choices and solutions to the challenging problem of controlling such a dynamic network from a central location. Our review is limited to works that consider networks which include mobile nodes in a mesh topology and does not include solutions for wireless mesh networks with static nodes or semi-static nodes (such as UAVs hovering over the same location). Two systematic review papers on the topic are [25] (which is however limited to Flying Ad hoc Networks (FANETs) and [26] (which considers only military Software-Defined networks of all kinds).
In our literature review, we classify related works based on the way SDN controllers and switches exchange CP messages. In-band (where CP messages are using the same wireless connections as DP messages) and out-of-band (where CP messages use some alternative wireless interface allowing for long-range, direct connectivity between controller and switch) are the two main approaches.
It is obvious that the out-of-band control assumes an uninterrupted connectivity of each switch with its associated controller regardless of topological changes in the data plane. In this case, the controller still needs to keep up with the high volatility of the MANET in order to provide fresh dataplane routes for new flows and quickly remedy broken datapaths. However, the stable control-plane assumption offers a significant advantage compared to the in-band control case: with in-band control when a DP route breaks and the sending node needs to contact the SDN controller to get a new DP route, it is quite probable that the known CP path to the SDN controller is also no longer valid. In addition, the connectivity to a controller might not be guaranteed at all times. In such cases, a switch that has lost connectivity to its controller will need to either resort to a non-SDN routing protocol, find and connect to another controller, or elect a new controller (collectively with other disconnected switches).
Thus, out-of-band control provides a robust and more efficient solution compared to in-band control at the cost of having to integrate two wireless interfaces on the same node. Alternative solutions either combine the use of in-band and out-of-band CP or place an SDN controller on every MANET node (or at least on a great portion of them) in order to minimize the latency and maximize the robustness of the switch-to-controller communication. Note that in this latter approach the reduction of control plane overhead in the SBI comes at the price of increased over-the-air inter-controller control message exchange.
Overall, we classify SDN-based MANET architectures in the following four categories: 1) Control Plane using out-of-band, single-hop communication between the SDN controller and switches. 2) Control Plane using multi-hop, in-band communication (same as the Data Plane). 3) Zero-hop or very-few-hops controller-to-switch connection (using a very large number of SDN controllers). 4) Hybrid network control, combining in-band and out-ofband control plane or a few central SDN controllers with controllers at every node. Table 2 summarizes the main attributes of the reviewed literature, discussed in more detail in the following subsections.

A. OUT-OF-BAND CONTROL PLANE
In [27], the authors investigate the main issues involved in realizing an SDN architecture for MANETs with real devices. They propose a practical implementation of the SDN paradigm with devices running a Linux operating system (OS). The considered devices have two wireless interfaces, one for the data plane and another one for the control plane. For the CP, a star topology is built in which each node is communicating directly to a centralized unit (CU) that is taking routing decisions in a centralized way, having full knowledge of the topology almost in real time. The proposed framework has two main advantages: (i) The overhead introduced in the ad hoc network is minimal, since each node just needs to be aware of its neighbors and communicate this information to the CU via the CP, and (ii) the complexity at each node in the network is radically reduced, because the routing algorithm runs at the CU. The paper presents experimental evidence comparing the suggested SDN routing protocol to the OLSR MANET routing protocol (olsrd-0.8.8). In both cases the nodes are implemented on Raspberry Pi devices. Open Network Operating System (ONOS) and Open virtual Switch (OvS) are used as SDN controller and switch, respectively. The presented results show that SDN significantly outperforms OLSR. Although real experiments are performed on a trivial 3-node network, it is conceivable that in the out-of-band control case, centralized SDN routing can outperform distributed routing protocols even for larger networks. This is also supported by the line of work reported in [28], [29], and [30]. The authors of these papers go as far as to claim that "without the assumption of an out-ofband, single-hop control plane, a purely centralized routing approach would be meaningless" [29]. In [28] and [29] a hybrid SDN-OLSR routing protocol is developed in which OLSR is used for short routes, while SDN routing is used for longer routes. Out-of-band control signaling is adopted and while in theory the control path can be multi-hop, for all simulation experiments it is assumed that every switch can reach the controller in a single hop. The results of simulation experiments (using ns-3) are presented in both [28] and [29] comparing the hybrid SDN-OLSR protocol with pure OLSR.
The main differences between [28] and [29] are: (i) in [28] nodes are placed on a fixed grid (no mobility) while in [29], half of the nodes move and (ii) in [28], MANET routing control packets are exchanged in-band which results to an unfair comparison with the SDN-OLSR hybrid protocol while in [29] out-of-band control signaling is applied to traditional MANET routing protocols. The simulation results presented in [29] show the hybrid SDN-OLSR protocol having a clear advantage over three of the most widely used MANET routing protocols (namely Destination-Sequenced Distance-Vector (DSDV), Ad hoc On-Demand Distance Vector (AODV) and OLSR routing) in terms of four common network performance metrics: packet delivery ratio, packet drop ratio due to route unavailability, average throughput, and signalling overhead. Simulation experiments were based on a MANET with 49 nodes using 802.11a PHY and MAC and moving around according to a Random Walk mobility model.
Offloading of cellular communications traffic through the use of an SDN-based Wireless Mesh Network formed by the smartphones themselves is described in [30]. Control plane information is exchanged via direct cellular links while data packets are sent either through cellular links or via a WiFi mesh network formed by the smartphones (up to 3 hops from an AP). The paper provides an architectural and operational description of the solution, while presented experimental validation is performed through a toy network scenario.
A single SDN controller, named the MANET-Controller (MC), which connects to all mobile nodes in the MANET through an out-of-band CP channel in star topology is also assumed in [31] while the DP follows multi-hop higher data rate paths through the MANET. An important difference with other related works is that the centralized routing algorithm in this paper is not a mere minimum-path algorithm but takes into account the requested bandwidth of the flow and the estimated channel utilization by network nodes. The MAC protocol used in the DP is Carrier-Sense Multiple Access with Collision Avoidance (CSMA/CA). An elaborate scheme is developed that allows each node to measure its channel utilization including time spent to transmit, receive, or compete for the medium. Each node periodically reports both its local topology (one hop neighbors) and its measured channel utilization to the MC so that the MC keeps an up-to-date Network Topology Graph (NTG). As per SDN norm, when a node wants to launch a new flow, it sends a request to the MC and the MC determines the min-cost route based on its centralized knowledge of the NTG, channel utilization, and flow requested bandwidth. If no route exists that can accommodate the requested bandwidth, the controller determines the maximum data rate that can be provided to the flow and informs the requesting node accordingly. The proposed solution is implemented in Omnet++ and evaluated through simulation experiments aiming at verifying that the main concepts of channel utilization estimation, resource utilization-aware routing, and requested bandwidth adaptation work as expected. Small networks with one or two flows and limited node mobility are used in the simulation experiments in order to provide clear explanations of the obtained results. As a consequence, the presented results provide a convincing proof-of-concept of the validity of the proposed design for small scale networks while scalability and robustness to high node mobility is not evaluated. Even so, the authors admit that CSMA/CA is a poor design choice suffering from increased latency when the network utilization is high. It also induces errors in the estimation of channel utilization. A TDMA MAC would be a better choice as the MC could provide synchronization and centrally optimized time-slot allocation.
A Software-Defined VANET architecture is described in [32], where VANETs are defined in a broader sense, i.e., vehicular networks in which "vehicles communicate with each other through Vehicle-to-Vehicle (V2V) communication in ad hoc fashion, and Vehicle-to-Infrastructure (V2I) communication through road-side-units (RSU) and mobile broadband (e.g., 4G/LTE)". The paper assumes out-of-band control with long range wireless connections, i.e., LTE/Wimax for the CP, and high bandwidth wireless connections, i.e., WiFi for the DP (in-band CP is left for future work). Despite the direct-connection CP, the authors still consider wireless CP connections to be unreliable and introduce 3 modes of operation in order to deal with this issue: Central Control Mode, Distributed Control Mode (where connectivity to the SDN controller is lost and a distributed routing protocol, such as Greedy Perimeter Stateless Routing (GPSR), takes over), and Hybrid Control Mode (where the SDN controller does not hold complete control, but instead can delegate control of packet processing details to local agents). One example of the Hybrid Control Mode would be that instead of sending complete flow rules, the SDN controller sends out policy rules which define general behavior, while the SDN wireless nodes and SDN RSUs use local intelligence for packet forwarding and flow level processing. The paper describes the advantages of centralized control in different networking layers (path selection, frequency/channel selection, transmit power selection) and specific VANET services that can benefit from enhanced network control. Simulation results (using ns-3 and the Simulation of Urban MObility (SUMO) traffic simulation package) show SDN based routing to exhibit higher packet delivery ratio than traditional MANET routing protocols (DSDV, AODV, OLSR, and GPSR). Additional simulation results illustrate the efficiency of the fallback mechanism from Central Control Mode to Distributed Control Mode and the benefits of centralized transmit power adjustment.
In [33], centralized SDN control is proposed as a means to boost the resiliency of FANETs against unreliable wireless channels or intentional jamming. The proposed architecture assumes that each UAV is equipped with various Radio Access Technologies (RATs) (such as LTE, 802.11ad, 802.11ac) to communicate with each other and the selection of which RAT to use in each hop is a part of the centralized path planning. Furthermore, all CP data flow through a dedicated control channel between the controller and UAVs (although it is not clear whether this is an out-of-band physical channel or a logical control channel over in-band physical links). Using a heuristic algorithm based on a novel 3D spatial coverage related metric, the SDN controller calculates diverse multiple paths among UAVs so that isolated and localized failures can be automatically addressed (by using an alternative path) without the need to contact the controller. Evaluation results reveal that the proposed multi-path routing algorithm reduces the average end-to-end outage rate while increasing the average end-to-end delay when compared to traditional multi-path routing algorithms. Details on the employed evaluation tool are not revealed. Note that the proposed algorithm is evaluated in networks with static nodes where some links fail (because of jamming) while the effects of node mobility on path stability are not considered. Further details on this line of work (such as the specifics of the multi-path routing algorithm and the tools used in performance evaluation) are provided in [34]. Additional simulation results are also presented in [34] which verify and strengthen the conclusions of [33].
Another SDN-based architecture for FANETs is proposed in [35]. More specifically, the work is considering Flying Ad hoc Sensor Networks (FASNETs) in which all the traffic generated at the UAVs of each network cluster is sent to a single UAV in the cluster (sink or gateway node). Therefore the goal of the routing algorithm is to construct a routing spanning tree. The paper proposes a hierarchical FASNET architecture: the network is organized into clusters with one local controller per cluster and a single global controller. However, the rest of the paper focuses on describing and evaluating a centralized, traffic-differentiated routing (TDR) algorithm for a single cluster. The algorithm differentiates flows according to their specific QoS requirements in terms of delay and reliability. Elaborated prediction models are proposed to measure the delay and transmission reliability of wireless links taking into consideration the nodes' position and velocities, and historical data about past packet transmission successes and failures. Based on all the above, an optimization problem that seeks to satisfy the QoS needs of different flows in the best possible way is formulated. As this is an NP-hard problem, an ant-colony based heuristic is proposed. Extensive simulation results compare the proposed TDR algorithm with the MintRoute algorithm which is the standard routing algorithm in TinyOS. The results show that TDR not only reduces the average delay for delaysensitive applications but also improves the data integrity for reliability-requisite applications. There is no mentioning of the simulation tool used nor any SDN-related implementation details provided.

B. IN-BAND CONTROL PLANE
Despite their above-mentioned dogmatic claim that centralized routing would be meaningless without an out-of-band, single-hop CP, the authors of [29] later proposed an SDN centralized routing scheme with in-band CP in [36]. With in-band control signaling, a switch cannot be guaranteed to have a connection to a controller at all times. This is addressed by introducing multiple SDN controllers to support network disconnections. In this setting, the paper considers the problems of: discovering network controllers; selecting to which network controller to connect when more than one are available; and selecting which switch to upgrade to controller status when no controller is available in a particular network partition. "Master" controller selection is driven by the hardware capacity of nodes measured as the maximum number of new flows per second that an SDN controller can support. Furthermore, a new way of combining SDN with MANET routing (more specifically the IETF standardized Babel routing protocol) is proposed. MANET routing is always on but it is overridden by SDN rules when a master SDN controller is available. In terms of implementation details, the Neon SDN framework for dynamic networks developed by the Laboratory of Integration of Systems and Technologies of the French Alternative Energies and Atomic Energy Commission (CEA LIST) is employed. Neon aims at enabling fast devices configuration and services deployment -within seconds -in dynamic and unconfigured infrastructure contexts. Performance evaluation results only concern measurements of controllers' capacity (using the Common Open Research Emulator (CORE) to emulate the Wireless Mesh Network and hardware in the loop to run the SDN controllers). No results about network Key Performance Indicators (KPIs) (throughput, packet loss rate, control overhead, etc.) are provided in [36].
A similar approach is taken in [37]: in-band control plane is assumed and an algorithm for elevating a mobile node status from switch to controller is described. When connectivity to the current controller is lost, a new local master controller is elected. Centralized routing decisions are taken by an SDN controller which is aware of the full network topology and flows characteristics. In addition, the paper considers delay-tolerant and non-delay-tolerant flows and treats them accordingly using Delay Tolerant Networking (DTN) principles: packets of delay-tolerant flows can be stored in intermediate switches until a suitable next-hop connection becomes available. Emulation experiments are performed based on the Mininet-WiFi emulation tool for network scenarios containing 9 UAVs and 9 ground vehicles all moving according to a Random Waypoint mobility model. Emulation results show better performance of the proposed SDN-DTN based routing compared to traditional MANET routing when multiple simultaneous video streaming flows are transmitted over the network. The paper does not specify which MANET routing protocol was used in the comparison.
The same research group has published another 2 papers [38] and [39] which are similar to each other. They both consider a tactical MANET including ground vehicles and UAVs. The ground vehicles are equipped with SDN switches controlled by a single SDN controller via in-band control plane while the UAVs form a wireless mesh network (it is not clear why they are not also part of the SDN network). UAVs are also the sources of video streams transmitted towards the lead ground vehicle in the formation. The papers present emulation results based on Mininet-WiFi and measuring KPIs on the received video streams (video startup time, number of video stalls, duration of video stalls, and a Mean Opinion Score which is derived from the other KPIs based on an analytical formula). These KPIs are measured for a varying number of simultaneous video transmissions and under different mobility models. In [38] the SDN approach is shown to achieve better KPIs than a mesh networking protocol (although the mesh routing protocol used in the comparison is again not specified).
The challenges of the in-band control approach are also discussed in [40] and some potential solutions are described on a more theoretical level. Three methodologies for coping with the high volatility of MANETs are presented: (i) when connectivity to the controller is lost, some of the nodes in the SDN network can dynamically migrate to a distributed routing protocol, (ii) a cluster-based hierarchical control based approach (MANET routing intra-cluster, SDN intercluster routing) is applied, (iii) a proactive approach in which backup SDN-forwarding rules are stored in the forwarding tables of the switches and used when the primary route is broken is followed (based on the stateful forwarding feature available in certain SDN protocols). To demonstrate some of these ideas, the authors have implemented a small-scale SDN prototype of a hybrid control architecture. Their prototype implementation consists of one Python-based Openflow controller (POX) and 3 OvS switches running on common smartphone and laptop devices. Their experiments suggest that by pushing a certain level of control logic to the mobile devices ("hybrid SDN control") a multi-fold reduction in failure reaction time is achieved compared to the fullycentralized OpenFlow system where the controller responds to all failures. The gains highly depend on the quality of the wireless channel between the controller and mobile devices. Unfortunately, the experiment is performed on a very small network using a generic "hybrid SDN control" scenario. Note that the three methodologies described above have not been fully developed, implemented, and tested on the prototype testbed.
The authors of [40] further elaborate on their proactive backup routes methodology in [41], providing mathematical analysis, as well as simulation and emulation results. The main concept is to proactively discover a sufficient number of backup paths to be used when the primary path breaks. The backup paths are stored in the forwarding tables of the switches and used under specific conditions of link unavailability (stateful forwarding). The authors propose the Message Dissemination Problem (MDP) pertaining to the minimization of the number of control messages carrying information on the state of wireless links. They formulate it as a mathematical optimization problem and develop a heuristic algorithm for solving the problem efficiently (although some potential network level optimizations such as message broadcasting are not considered). Network simulation and emulation (using Mininet) results compare the performance of this "hybrid SDN" technique with traditional SDN, while no comparison to distributed MANET protocol is offered. A real-world tactical ad hoc network dataset with over a hundred nodes is used in these experiments. Obtained results demonstrate an order of magnitude lower control overhead and a 50% increase to the data packets reaching their destination.
After describing different network architectures, [42] focuses on comparing an in-band, centralized SDN control architecture with a traditional MANET using the AODV routing protocol. As the paper puts special emphasis on cybersecurity, in the MANET configuration AODV is complemented with iptables for packet filtering, while in the SDN architecture the inherent SDN flow control plays the role of denying flows based on security considerations. In both cases, additional traffic and application monitoring utilities can enhance the security of the network. The paper provides valuable design and implementation suggestions: the use of AODV as a routing protocol in the control plane and as a means of network topology discovery (instead of using a protocol like Link Layer Discovery Protocol (LLDP) as in a typical SDN network), the extension of SDN SBI using the experimental extension feature of OpenFlow, the separation of routers and hosts in two separate IP networks which allows for an easy identification of flows, etc. Experimental evaluation is based on a 4 personal computers (PCs) "testbed" (3 PCs represent UAVs and 1 represents the ground control station (GCS)) interconnected through a wired network. Realistic UAV-GCS data (telemetry, command & control, and live video feed) captured from an actual UAV mission are used in the evaluation experiments. Experimental results show that the SDN approach performs slightly worse in terms of route discovery latency and control traffic overhead compared to AODV, even in this tiny network. This is a small price to pay in order to use SDN's capabilities in improving network security.
A hybrid architecture is proposed in [43] with the idea of splitting the routing decision logic between the controllers and the data plane nodes. Specifically, the controllers can break the routing path into segments, similar to the segment routing technique, and broadcast the list of segment labels to the data plane nodes. The latter are able to make the actual forwarding decisions for each segment in a distributed manner, e.g., by running an existing MANET protocol like OLSR. The authors claim that this hybrid routing approach can: • improve reliability compared to the conventional SDN architecture (since nodes can neglect the controllers routing guidelines if the intermediate nodes cannot be found by the distributed protocol and the controllers are not reachable to update the guidelines) • improve performance compared to the current MANET protocols (as the SDN controllers can force the packets to go through specific nodes with special roles and middle-box functionalities, as well as indirectly loadbalance the network).
The paper presents some experimentation results based on a 5-node testbed, which motivate the proposed architecture. The implementation and evaluation of this architecture (along with the resolution of a number of discussed open issues) is left for future work. A MANET/FANET comprising of a single SDN controller, multiple "relay nodes" (whose position can be controlled in order to improve network coverage and performance) and multiple "independent nodes" (whose position cannot be controlled) is considered in [44]. The paper proposes algorithms for optimizing the controller and relay nodes' placement (referred to as "topology management"). Topology management is a combination of two different phases: "construction" (computationally expensive initial solution) and "adjustment" (of the solution to perturbations of the network state). When the adjustment is not enough to maintain a satisfying network performance, construction is performed again (this is decided by an "integration" algorithm). The paper proposes a UAV communication protocol based on [45] and topology management algorithms based on [46]. The "construction" algorithm employs Particle Swarm Optimization (PSO) to approximately solve a very hard optimization problem while the "adjustment" algorithm is gradient based. A time complexity analysis of these algorithms is provided along with simulation results showing the dependence of network KPIs (packet loss, latency, percentage of connected time) on the number of relay nodes, under the proposed approach.
A comprehensive solution aiming to achieve low overhead and high performance control message dissemination in an SDN-based FANET is described in [47]. The proposed network architecture is cluster-based with the clustering algorithm being executed at the global controller. In this algorithm the number of clusters is computed using the Gap statistics method while a Fuzzy C-means method determines the cluster head (sort of local controller) and members for each cluster. In addition, prediction algorithms are used by the global controller to forecast essential information (such as UAVs' current location or remaining energy) based on past values so that the frequency of status updates by the UAVs can be reduced. Control message aggregation at cluster heads also contributes to overhead reduction. Extensive simulation results are provided to show the gains of this approach in terms of control messages Packet Delivery Ratio and overhead, and the average energy consumption compared to existing SDN-FANET architectures.
Distributed SDN in wireless networks consisting of multiple inter-connected network domains, each managed by one SDN controller, is studied in [48] from both experimental and analytical perspectives. Both controller-to-switch (Southbound) and controller-to-controller (East-/West-bound) communication takes place in-band, through WiFi multi-hop links. The paper presents experimentation results (using a very small test-bed network with 4 nodes) including the control message latency distribution for 0, 1, and 2-hop paths as well as the effect of controllers' placement on the average latency of such messages (the exact same results are also presented in [49]). It also provides emulation results of control overhead traffic using Mininet as an emulation tool and ONOS SDN controllers. Based on the above overhead and latency results, the authors conclude that it is challenging to achieve full status synchronizations among controllers in a real system. They proceed to analyzing and quantifying how intra-/inter-domain synchronization levels and network structural properties affect the performance of distributed SDN architectures. They then establish analytical methods for performance estimation based on a generic network model under four canonical inter-domain synchronization scenarios. Specifically, they first derive an asymptotic expression to quantify how dominating structural and synchronizationrelated parameters affect the performance metric. Then they provide performance data analytics for an important family of networks, where all links are of equal preference for path constructions. Finally, they establish fine-grained performance metric expressions for networks with dynamically adjusted link preferences. These theoretical results reveal how network performance is related to synchronization levels and intra-/inter-domain connections, the accuracy of which is confirmed by simulations based on both real and synthetic networks.
A hierarchical, self-organizing SDN architecture for mobile tactical edge networks (MTEN) is described in the short paper [50]. The network is dynamically partitioned into multiple temporary domains in a self-organizing manner. In each domain, a certain node is appointed as a local SDN controller. The described protocol includes: (i) a simple neighborhood discovery mechanism, based on HELLO messages; (ii) a distributed game-theoretic network partition algorithm, not adequately explained due to space limitations; and (iii) a brief description of two functions: "user handoff" (from one controller to another) and "controller substitution" (a switch takes the role of controller). The protocol design is work-inprogress and its implementation and experimental evaluation is left as future work.
Another paper assuming in-band control combined with a more comprehensive approach to flow management is [51]. The paper describes an architecture and an associated prototype implementation for SDN-based QoS management in MANETs, which, according to the authors: (i) allows high flexibility via deployment of new flow management policies at provisioning time, (ii) properly handles node join/leave events, and (iii) determines MANET "islands", usually managed by separated SDN controllers. The SDN controller can take better routing and traffic engineering decisions based on knowledge not only of the topology and network status, but also of application requirements, such as flow priority or unicast/multicast communication. Although all the above concepts are capturing essential design considerations for SDN controlled MANETs, the paper does not take into account key properties of the wireless medium such as a transmitter's ability to broadcast a packet to multiple receivers and the interference caused by a transmitter to unintended receivers. At the same time, experimental results are limited to a network of 3 nodes in which N1 and N2 are connected via IEEE 802.11g, while N2 and N3 via Ethernet. Obviously, such experiments cannot evaluate the proposed solution's scalability or ability to handle node mobility and join/leave events that are causing network topology changes and/or controller unavailability.
Although the primary focus of [52] is in presenting a testbed for tactical SDN experimentation, the network archi-tecture of the testbed clearly assumes in-band East-/Westbound and Southbound control message exchange. The paper also presents a tactical network use case which is expected to benefit from SDN's centralized and cross-layer control: adaptive bit-rate streaming from a number of fixed video cameras trying to track a moving vehicle. The use case and emulation setup is presented in detail, however quantitative evaluation results are not provided.

C. MINIMUM DISTANCE CONTROLLER-TO-SWITCH COMMUNICATION
The main issue with in-band control is the loss of connectivity of individual switches with the controller and the need to discover a new route to it. Associated with multi-hop connections is also the issue of latency in control message exchange.
To overcome these problems, it has been proposed to place SDN controllers at every MANET node or at least at a large portion of MANET nodes so that a switch can communicate with a controller either over a wired connection (inside the node) or wireless connection with very few hops (single hop would be ideal of course, if possible). Each one of these controllers maintains a local picture of the global state of the network and makes centralized routing decisions for new flows (or flows that need to be rerouted) which originate from one of their associated nodes. However, the combined decisions of different controllers might not be optimal as a whole. For this reason, it has been proposed that the controllers communicate through their East-West interfaces in order to maintain the same view of the network and coordinate their decisions. Note that this multiple-controller design approach to SDN-driven MANETs is not to be confused with the need to use more than one controller for scalability reasons, when the network becomes too large.
A small controller-to-switch distance approach is taken in [49] where the decision on the number of SDN controllers in the network and the nodes on which they are located is formulated as a mathematical optimization problem. In theory, this covers the full range from having only one controller in the network to having one controller per network node. However, it is evident from the problem formulation that the goal is to place controllers in a good portion of network nodes in order to keep the switch-to-controller paths to a small, manageable length. Note that the paper considers a mixed network topology comprising of wired, infrastructure-based wireless, and ad hoc wireless segments. The paper presents the same experimentation results as in [48]. It then provides very similar emulation results with the ones shown in [48] with the notable difference that it uses two types of SDN controllers: ONOS and OpenDayLight. The reported results pertain to controller-to-switch and controller-to-controller traffic for varying number of controllers and switches in the network and under both a leader-based and a leaderless synchronization strategies. The paper then proceeds to formulate the controller placement problem as a constrained minimization problem and propose approximation solutions which perform close to optimal and substantially better than state-of-the-art methods as evidenced by extensive evaluation results. Last but not least, a thorough analysis of the tradeoffs between control message latency and overhead, and of the interplay between various performance and reliability objectives is provided.
In [53], the authors propose an SDN-based approach to support multi-domain heterogeneous MANET interconnection where each MANET can use its own legacy distributed routing protocol (such as OLSRv2, Open Shortest Path First-MANET Designated Router (OSPF-MDR), Babel, etc). Their approach builds on a three-tier hierarchical network architecture: Upper-Tier Network Level, Mid-Tier Network Level, and Lower-Tier Network Level, to seamlessly interconnect these heterogeneous MANETs. The Upper-Tier Network is where the Network Operation Center (NOC) running SDN management applications is located. The Mid-Tier Network (MTI) Level consists of "MANET vehicles", each one of them containing an SDN controller, a Gateway (GW) router and a number of MTI routers. The GW router is responsible for connecting to the Upper-Tier Network and to other GW routers. Each MTI router connects with a single Lower-Tier Network which is one of the heterogeneous MANETs mentioned above. Routing between GW routers is performed using the Babel routing protocol while routing inside each Lower-Tier network uses its own legacy routing protocol. Thus, the role of SDN controllers is to translate the information between different routing protocols in order to achieve inter-domain routing. Obviously, in the MTI network, all nodes contain an SDN controller which is connected to the "routers" via wired connections. The paper proposes a discovery protocol to automatically discover the lower-tier networks and seamlessly connect them to the rest of the network. The proposed SDN-based approach is implemented using the emerging Constrained Application Protocol (CoAP) as the Northbound and Southbound APIs. Performance evaluation experiments have been conducted on a CORE-based emulation environment of 48 nodes. However the presented experimental results (in terms of overhead, packet loss ratio, and Round-Trip Time) are mostly proofof-concept results and do not present a comparison with alternative designs or provide any insight on the dependence of these performance metrics on varying network parameters.

D. HYBRID CONTROL PLANE ARCHITECTURES
There is a small number of works in the literature proposing architectural designs that cannot be categorized in any of the above 3 categories. These are "hybrid control" architectures combining out-of-band and in-band CP and/or centralized and distributed control. One motivation for using a combination of in-band and out-of-band CP is the following: MANET nodes might be equipped with more than one wireless interfaces with different usage costs (e.g., free WiFi links for node-to-node communication and cellular longrange links with metered data service). In this case, the DP and CP are both utilizing the low-cost connections and only when connectivity through the low-cost network is lost, the CP reverts to the higher-cost network to quickly establish communication in the SBI.
The Temporospatial Software-Defined Networking (TS-SDN) architecture proposed in [54] is Google's approach to building a network of stratospheric balloons to extend LTE access into areas without fixed wireless infrastructure. The idea is to mount LTE eNodeB's on these stratospheric balloons which create a mesh network with highly-directional links to relay traffic to an Evolved Packet Core (EPC) in the Core Layer of the network. Unlike other solutions which mount eNodeB's on multi-copter drones that can stay almost static above a given location, stratospheric balloons constantly move thus making the network topology highly dynamic. However, in TS-SDN, the SDN controllers utilize knowledge of the physical position and trajectory of each aerial platform and its antennas to make predictions about the future state of the lower-level network. So, in contrast to traditional MANET protocols, centralized routing in TS-SDN can update routes proactively in anticipation of future route failures. In fact, simulation results using ns-3 with a large network (487 UAVs and 38 ground stations) illustrate the delays of setting up routes at start up and after route failures for 3 popular MANET protocols (AODV, DSDV and OLSR). These delays are completely eliminated by TS-SDN. Nevertheless, the paper suggests to use a distributed routing protocol in case a UAV node is disconnected from the controller due to an unpredicted link failure and a lowdata-rate, out-of-band CP channel (such as satellite control and non-payload communication (CNPC) links) as an expensive solution of last resort. Google has developed their own Southbound protocol (they call it Control Data Plane Interface (CDPI) protocol) as they found existing "CDPI" protocols to be insufficient for TS-SDN.
A novel SDN architecture for Airborne Backbone Networks (ABN) is proposed in [55]. The architecture differs from traditional SDN as it splits the control layer into two sublayers: in the upper control sublayer there exists one main "ABN controller" which takes central decisions (although other backup controllers are also provisioned) while in the lower control sublayer one "platform controller" is assigned to each (airborne) wireless network node. The platform controller acts as a proxy for the ABN controller but has also distributed network control capabilities for when the centralized network control mode is invalid or inefficient. An OSPF extension supporting Segment Routing (SR) is used in the CP in order for the controllers to gather network state information, while routing in the DP is based on Multiprotocol Label Switching (MPLS). In addition, the control channel employs omnidirectional communications, while transmissions within the data plane adopt directional communications. The paper presents the system design in great detail. It then proposes a network traffic scheduling algorithm to improve the transmission reliability and bandwidth utilization by balancing network traffic to multiple reliable transmission paths. Theoretical analysis proving the correctness of the algorithm is presented along with simula-tion results (using the network simulator EXata 5.1) showing performance improvement compared to alternative solutions.
A network architecture combining SDN and Message Queue Telemetry Transport (MQTT) in FANETs is proposed in [56]. The aim is to achieve a hybrid network structure with dynamic, reconfigurable capabilities, as well as lightweight, distributed features that are suitable for networking a swarm of UAVs in military applications. The full network can contain multiple swarms of UAVs, as well as air, naval and ground forces interconnected by radio, satellite and delay tolerant (employing UAVs as data mules) communications. The control mode is hybrid with an elected master controller and all other UAVs in a swarm acting as "slave" controllers. UAVs implement MQTT to publish and subscribe for application and control flows. In order to integrate SDN and MQTT, some MQTT interfaces have been added to the SDN controllers in order to expose all the key functionalities of a UAV (including position, flight information, remaining energy, node connectivity, channel quality, etc.) The paper presents the proposed complex architectural design in great detail (including full-fledged functional diagrams of the master and slave controller) and introduces a QoS-based multipath routing framework to calculate disjoint optimal paths between sources and destinations. However, no evaluation results or any mentioning to an actual implementation of the proposed design are provided.

E. REMARKABLE ISSUES ABOUT SDN FOR MANETS
• Centralized network control, one of the key concepts of SDN, seems to be in contradiction with the distributed, highly dynamic, and ad hoc nature of MANETs. Nevertheless, researchers have been trying to overcome this contradiction and introduce SDN in MANETs in order to bring the recognized benefits of SDN into this domain. • The most efficient way of doing this is to consider an out-of-band CP, offering a direct and robust connection between the centralized controller and all the switches in the network. This is not a new concept as there have been proposals to use low-data rate, high communication range wireless interfaces for controlling MANETs even in the pre-SDN era (e.g., cellular CP with WiFi DP [57] or using wireless links at 5 GHz for the CP and wireless connections at 60 GHz for data exchange [58]). As expected, this approach is shown to improve network performance compared to traditional MANET routing protocols. • However, the need to support two wireless interfaces on the same node increases the cost of the out-of-band solution and even makes it prohibitive in the case of Size, Weight and Power (SWaP) limited devices. A long-range CP can also create security risks, especially in military environments where the enemy can detect and localize the source of long-range transmissions. For these reasons, a lot of effort has been devoted recently in solutions featuring an in-band CP. The results of this effort verify that when the objective of the routing algorithm is finding minimum-hop paths, centralized routing can only hope to be a little worse than distributed routing due to the delay and overhead of collecting network topology information at the controller. When centralized routing takes additional considerations into account (such as the traffic demand of flows and availability of network resources, finding disjoint alternative paths, or finding robust paths through predicting the future availability of links), it can turn out to outperform traditional MANET protocols in many important KPIs. • In order to overcome the limitations of an in-band control plane, a number of works have proposed alternative approaches which attempt to design efficient, scalable, and robust SDN-based network architectures for MANETs. They either propose an in-band CP operation which falls back to an out-of-band mode when needed or the minimization of the switch-to-controller path lengths by placing SDN controllers into (almost) all mobile nodes. This later idea minimizes the SBI messages transmitted over wireless links, at the cost of increasing the over-the-air CP messages exchanged between SDN controllers. • In conclusion, it is widely recognized that traditional SDN protocols and implementations are not adequate for MANETs and therefore new solutions had to be developed or the existing ones appropriately redesigned and amended in order to build a high performance SDNbased MANET. At least one commercial deployment of such a network is now in operation [54] and we believe that more are going to follow.

IV. SDN-SDR COEXISTENCE: BACKGROUND, DRIVERS AND TECHNOLOGY
In this section we present the background, drivers and technology behind the efforts for hosting SDR and SDN in a single wireless device. Initially, a brief introduction to the SDR basics along with the latest SDR standardization efforts and the significance of the SDR for the wider softwarization efforts are presented. Next, the drivers of the coexistence of SDR and SDN in radios, especially military radios, are discussed followed by a literature overview of combined SDN-SDR architectures. Finally, a generic interaction loop between SDN and SDR is presented. Before we proceed it must be noted that SDN and SDR technologies can be simultaneously hosted in the same device in the following two modes: • Non-interactive mode: in which the two technologies are completely independent; in this mode, the SDR acts as a PHY and MAC protocol engine on top of which higher layer protocols are served by a general purpose processor running the SDN controller and the switch software. • Interactive mode: this is the case of interest for this article because it allows cross-layer design and optimization of the network. In this mode, certain elements of the PHY and MAC layers of all the nodes of a wireless network can be advertised and controlled (modified) in an orchestrated way throughout the network by using the SDN protocols. A possible SDN-SDR interaction loop which supports this mode is presented later in this section.

A. SDR AND CURRENT SOFTWARIZATION EFFORTS
SDR is now a mature technological framework that seems to have initially been conceived during the mid-1980s by a research team of E-Systems Inc (now Raytheon), [59]. Its description, main concepts and components were originally published a few years later, in the early 1990s by J. Mitola in [60] and [61].
The main objective of SDR was (and still is) to eliminate the need for specialized hardware by introducing software modules that implement as many communication functions as possible. Thus, the same transceiver can be used in a completely different telecommunication system by loading the corresponding software modules. In an ideal SDR wireless device, as depicted in figure 2, the received signal is amplified and down-converted by the Radio Frequency (RF) subsystem, converted to digital by the Analog to Digital Conversion (ADC) subsystem, while all the subsequent processing in order to retrieve the transmitted data is done in the digital domain by the Processing subsystem (PS). The PS can be any processing unit that has the computational power to process, in real time, the received digitized signal. The ideal PS would be a General Purpose Processor (GPP), being the easiest to set-up, program, and use. A more realistic approach involves the use of at least one Field Programmable Gate Array (FPGA) platform in order to handle the real-time requirements of most transceivers.
The initial motivation for designing, building, and deploying SDR technology in the field, was to improve the interoperability between different communication systems of the US armed forces and ease the corresponding logistics. Nevertheless, the promise of minimizing the hardware dependencies which leads to maximizing reconfigurability, was interesting for civilian applications as well [62].
SDR found its way towards the standardization bodies, initially through the SDR Forum, followed by the Wireless World Research Forum (WWRF) and currently ETSI. These bodies issued [63] and still issue to this day under the umbrella of the Reconfigurable Radio Systems (RRS) Technical Committee (TC) [64], various white papers, reference models, architectural approaches and Technical Specifications (TS).
ETSI's Software Radio Reconfiguration model is described in a recent ETSI white paper [65] and in the EN 303 146 series of European Standards [66]. Its target is to enable device manufacturers to gradually implement software reconfigurable radios over different generations of equipment. Various use cases have been introduced, while the key challenges that have been identified are reproduced below from [65]: • Problem Statement 1: How to transfer and install radio software components to a target platform in a secure way. • Problem Statement 2: How to enable a user to access new software components. • Problem Statement 3: How to deal with device certification in the context of novel radio software components. • Problem Statement 4: How to achieve software portability and execution efficiency. • Problem Statement 5: How to enable a gradual evolution towards software reconfigurability.
Solutions to the aforementioned problems follow a gradual, step-wise approach from partial to fully flexible software reconfiguration, while efforts are made to produce a framework which is compliant to the current network virtualization context [65], [67]. These latest standardization and research efforts, such as those presented in [68], [69], [70], [71], and [72], indicate that SDR has become part of the broader "softwarization" endeavors that relate to the design and deployment of 5G networks. The term softwarization is used in many domains and can be defined as "moving functionality from hardware to software", [72]. The main objective of softwarization is to improve the flexibility of the networks that will need to support a wide spectrum of services offered to wireless devices that are no longer as homogeneous as they used to be in the previous generations of cellular networks. In [69], the integration of Network Function Virtualization (NFV), SDN and SDR is examined in terms of standards, necessary standardization efforts and open research problems while a high-level architecture combining all three technologies is suggested. In [71], a software defined radio virtualization layer, called HyDRA, that enables the execution of multiple programmable air-interfaces on top of one RF front-end is presented. The authors of [72] expand the work of [71]; the term Virtual Radio Function is introduced, while the well-known terms "Virtualization" and "Orchestration" are defined in the context of SDR. Both papers relate to the concept of radio virtualization that was introduced in [73]. In the latter paper, radio virtualization is defined as the process of sharing and allocating resources belonging to a physical radio link, while the term Virtual Radio (VR) is used to describe the resources that are used by a particular virtual network node. All kinds of resources can be shared, including spectrum, Physical Radio Blocks (PRB) in LTE, or even the RF front-end, [71]. Of particular interest for the SDR technology is the case where multiple Virtual Radios can coexist and share the same RF front-end, while providing some isolation of computational and communi-  cation resources. A high-level architecture of a virtualised SDR platform in which several SDRs are using the same RF front-end is presented in [71] and reproduced in Figure  3. Similar to computer virtualization, the SDR hypervisor abstracts the physical radio device into VR devices and schedules resources available at the physical radio between different VRs. The SDR hypervisor must be able to ensure the viable and reliable coexistence of multiple VRs as well as their isolation, so that any configuration or misconfiguration of a particular VR will not affect and interfere with other coexisting VRs. The HyDRA SDR virtualization platform is using the well known Universal Software Radio Peripheral (USRP) platform [74] to provide the common RF front-end, while the SDR hypervisor is implemented on a single PC.

SDR
There are countless and comprehensive published articles and books that cover all SDR aspects, from politics to technology, from theory to product development, from software to hardware. The reader who wants to further investigate the origins, main concepts and applications of SDR can refer to the following indicative articles [1], [75][76][77][78][79][80][81][82][83][84][85][86]. The cited articles that relate to SDR are summarized in Table 3.  [67] Civilian standards and systems 2000 onwards The SDR-related standardization efforts for civilian mobile networks, including the latest ETSI standards.

onwards
The latest SDR trends including interaction with SDN and broader 5G efforts. Description of the SDR role in existing testbeds.

2009, 2015
Introduction to SCA, one of the major technologies that allowed commercialization of SDR to the military domain.

B. DRIVERS FOR SDN-SDR COEXISTENCE: THE TACTICAL RADIOS CASE
From a technical point of view, the benefit of combining the two technologies (in an interactive mode) in wireless networks is quite obvious as described in section I. In short, the combination of SDR and SDN in wireless nodes can lead to optimum cross-layer designs and greatly improve their overall performance. Nevertheless, we need to keep in mind that, usually, technical benefits need to be backedup by business incentives in order to be adopted by the corresponding industry. Such a case is presented in the tactical radios and MANET market. Although not all man-portable and vehicular tactical radios are SDR-based, it is clear that SDR hardware is now well established in the tactical domain and seems rational to assume that in the near future will prevail over classic hardware solutions; according to [91], approximately 200,000 SDR-based tactical radios are sold globally per year, a trend documented between 2008 and 2015. The main SDR market drivers are: (a) transition to Network Centric Operations and (b) increase interoperability by using suitable waveforms 1 .
Both US and European armed forces support the use of SDR technology in their radios through extensive, longterm, national and international programs. In USA, the wellknown Joint Tactical Radio System (JTRS) program [77], that sparked the commercial deployment of SDR technology in the 1990s, is now replaced by the Joint Tactical Network Center (JTNC) program, the role of which is to provide wireless software defined tactical networking solutions to 1 In military communications the term Waveform was originally used to describe the Physical Layer of a wireless telecommunication system. Overtime, the term expanded to cover everything up to the network layer and in some cases the term waveform is used to describe all the protocol layers of a military telecommunication system below the application layer.
US joint forces and to coalition forces [91]. In Europe, the European Secure Software Defined Radio Referential (ESSOR), is an initiative that started in December 2008 and is still active today. The main scope of this project was ". . . to provide architecture of Software Defined Radio (SDR) for military purposes and a military High Data Waveform (HDR WF) compliant with such architecture, thus offering the normative referential required for development and production of software radios in Europe. In addition, the project will deliver guidelines which are related to the validation and verification of waveform portability and platform reconfigurability, setting up a common security basis to increase interoperability between European Forces." [93], [92]. The ESSOR products are based on the Software Communication Architecture (SCA) [78] developed by JTRS in order to facilitate waveform portability amongst heterogeneous military SDR platforms. Besides ESSOR, in Europe there are several smaller national programs related to military SDR which are listed in [86].
At the same time, SDN is becoming popular in military strategic and tactical networks, especially coalition networks, where increased agility, flexibility, central control, dynamic policy definition and security are of paramount importance [26], [24], [94], [37]. The European Defence Agency (EDA) has identified nine technical research areas considered as requiring specific defence research and technology investments which have been formulated as Technology Building Blocks (TBB). One of these nine TBBs is SDN as can been seen in [95].
It is clear from the above that any new tactical radio and MANET design should seriously consider not just the coexistence but also the joint optimization of the two technologies, as SDR is well established in the field, while SDN gains popularity and has already been investigated and tested by various defence organizations.

C. LITERATURE OVERVIEW ON COEXISTING SDN-SDR ARCHITECTURES
In this section, we present a review of the SDN-SDR coexistence literature, which is currently very limited. It must be noted that this short review only examines papers that focus on the interactions of the two technologies with the purpose of cross-layer optimization. It does not deal with articles that refer to systems where both technologies are used but do not really interact or have any inter-dependencies.
An implementation of a cross-layer architecture based on the SDN-SDR combination is presented in [87]. The authors address the challenge of using real-time radio network information to improve service operation. To this end, they combine SDR and SDN on a Mobile Edge Computing (MEC) platform to jointly use radio and virtualization resources. Their findings suggest that the architecture under investigation allows the improvement of the service operation under a cross-layer design. The authors implement their proposed architecture, named Multi-Radio Edge Node (MUREN), on a testbed which is based on USRP and GNU radio for the SDR part, and on the Open vSwitch for the SDN part. The MUREN architecture consists of four parts: the waveform/radio, the virtualization platform, the application, and the MEC controller. This architecture enables crosslayer decision making, allowing the collection of information from several protocols layers. Specifically, the waveform part measures and collects Channel State Information (CSI) on each available waveform. This information (e.g., received signal strength, estimated radio throughput, queue size) is gathered by an SDR controller which then exposes it to a central MEC controller. The virtualization platform part is responsible for managing and forwarding the traffic from/to the applications and the radio interfaces, it is supported by an SDN switch and configured by an SDN controller. The latter also monitors the network interface of each application and measures throughput, packet error rate and jitter. All this information is transferred from the SDN controller to the MEC controller, and decision-making takes place for actions (e.g., flow rerouting) either on the programmable radios or on the virtualization platform.
The authors of [99] focus on autonomic network management optimization for efficient channel allocation in Cognitive Radio networks, leveraging the SDN-SDR combination via NFV virtual utility functions. The authors have implemented and evaluated the proposed approach on a real SDR testbed that exploits the capabilities of the ORCA 2 project infrastructure. The framework focuses on the three lower layers of the protocol stack, which are extended to include a vertical cross-layer interconnection. The physical layer is enhanced with SDR features such as spectrum sensing. Signaling between mobile nodes is exchanged over the SDN control plane. The SDN controller, in cooperation with other network 2 https://www.orca-project.eu/ entities, utilizes this information in order to determine the most efficient channel allocation. This is a representative use case where the SDN-SDR coexistence acts as an enabler for the implementation of a cross-layer optimization algorithm.
In [68], the authors describe and discuss the benefits of SDN-SDR coexistence in the context of LTE self-organizing networks (SONs) which require flexible network architectures with high degree of programmability. They propose a cross-layer architecture for a SON management framework, which leverages the symbiosis of SDN and SDR techniques. Under the scope of LTE, they describe the integration of SDN and SDR in an architecture that exposes interfaces to retrieve, exchange and update system configurations, and to supply and query context information in an automated way. The core of their proposed architecture is a cross-layer design of a Programmable evolved Node Base station (PeNB). The PeNB, that can be deployed in LTE networks and is equipped with an SDR-based multiprocessor platform, contains dedicated units for the processes of the three lower layers of the protocol stack (L1, L2 and L3). A virtualization layer abstracts the hardware resources and allows the provisioning of multiple virtual eNBs, which can be provisioned, controlled, and inspected as needed.
A generic architecture, named CrossFlow, which leverages the SDN-SDR coexistence is presented in [97]. In this work, the SDN model is extended by providing support for highly configurable software-defined radios. The authors implement the proposed architecture on a testbed based on GNU Radio and USRP for the SDR part, and on an OpenFlow interface for SDN that they developed. This interface has a key role for the proposed architecture and runs on the USRP platform. It allows the SDR platform to communicate directly with the SDN controller using OpenFlow messages, thus allowing for real-time reconfiguration and optimization of the protocols that run on the SDR platform. This cross-layer design allows SDN applications to obtain a centralized network view and enables online control of the PHY layer through an OpenFlow-compatible interface.
The work in [98] is related to Radio Access Network (RAN) management for 5G networks, and presents another example of a cross-layer design that is established with the combination of SDN and SDR technologies. The authors present and evaluate, using hardware testbeds, a Cross Layer Controller (CLC) which monitors in real time radio environment conditions as well as the transport network traffic. The CLC enables cross-layer resource management and dynamic adaptation. The exploitation of SDN and SDR techniques under the framework of the proposed cross-layer architecture allows a unified management in both the control and data planes. The radio resources in this work are abstracted as a resource pool, which is managed by the SDR controller. In a similar manner to the control plane, network resources (i.e., transport network links) are also managed by the SDN controller. The authors conclude that the coordination of both the SDR and the SDN controllers through the proposed CLC, optimizes network performance on dense radio environments IEEE ACCESS, Vol. , No. , month year Another work focused on 5G networks is [89]. The proposed design comprises two main parts, the SDN and the SDR layers. Their coexistence is exploited by establishing a cooperation regime between them, such that the SDR has access to various parameters of the radio environment, while SDN can use these results to determine a specific policy. The two layers are monitored and are under the administration of a Cross Layer Controller (CLC). This module simultaneously receives information from both layers in order to make a trade-off analysis of the layer conditions. In the suggested workflow, the CLC allows the access or suggest switching to a specific band. Also, in a dynamic network environment the CLC can make self-adaptive adjustments under a crosslayer policy, utilizing information from lower protocol layers through SDR and from upper layers though SDN.
Finally, the authors of [96] in the framework of the European project ORCA, investigate the necessity and feasibility of extending the virtualization of wireless networks towards the radio hardware. An SDR architecture is presented for radio hardware virtualization so as to facilitate softwaredefined wireless networks design and experimentation. The authors suggest that by adopting the virtualization-oriented hardware accelerator design, an all-layer optimised network performance can be achieved. Table 4 summarises a number of SDN-SDR coexistence articles with emphasis on the type of platform that the authors used, as well as the main technologies that were tested in the corresponding experiments.

D. A POSSIBLE SDN-SDR INTERACTION LOOP
In this subsection, we demonstrate a possible interaction loop between the SDR and SDN components that operate on a single wireless device. The purpose of this interaction is to exchange all the necessary information that will allow crosslayer optimizations.
In an SDN network, an SDR-enabled node may operate either on an active or a passive interaction mode. These terms are defined as follows: • Active interaction mode: a node reconfigures some of its PHY layer parameters. The reconfiguration is the result of instructions that were issued by higher layer protocols or the SDN controller or a combination of both. In other words, SDR is reconfigured in accordance to instructions that were received by using the SDN protocols. • Passive interaction mode: a node is using its RF frontend as a sensing device in order to acquire information about the network environment and then pass this information to higher protocol layers and/or the SDN controller for further processing. This information and the corresponding processing may result to the transition of a node from the passive to the active node. One possible scenario involving the aforementioned modes in an SDN-SDR interaction loop comprises five generic steps and is depicted in figure 4: 1) A wireless SDN switch, performs measurements on the channel (passive mode). The trigger for these measurements is not depicted in the figure but can be either preconfigured in the switch or initiated by the controller. 2) The measurements are transmitted to the (wireless) SDN controller using Openflow messages. The corresponding messaging scheme must be carefully designed in order to avoid protocol message overhead, and at the same time keep the controller(s) up to date in terms of channel conditions. 3) The SDN controller evaluates messages from the corresponding switches and may decide to instruct them to reconfigure some of their PHY parameters. It is safe to assume that the reconfiguration should be orchestrated in order to avoid having one or more switches operating under incompatible L1/L2 configurations.
4) The controller sends the reconfiguration commands to all switches using Openflow messages. 5) The switches decode the message and proceed to the corresponding reconfiguration (active mode).
The aforementioned SDN-SDR interaction scenario may occur in a wireless network (MANET or other) and can be supported by an SDN wireless node that uses a typical SDR architecture such as the one depicted in figure 5. In this architecture, L1 and part of L2 protocols are implemented on a computational platform that can support real-time operations, while the remaining L2 functionalities (if any) and all layers from L3 upwards can be implemented on a General Purpose Processor (GPP). The interaction between SDN and the L1/L2 protocols will require an appropriate interface which can be a typical API. All required messages of steps 1 and 5 of the SDN-SDR reconfiguration loop depicted in figure 4 will be exchanged through this API.
Important elements in the above loop, that should be carefully considered, are the type and size of the information exchanged between the nodes, the corresponding frequency of transmission, the SDN protocol messages that will be used in order to carry this information, the various optimization algorithms that the controller will use, and of course the network-wide orchestration of a possible reconfiguration. It is not our intention in this paper to study in detail the SDN-SDR interaction loop but rather to point-out that it must be carefully designed in order to assist cross-layer optimization; otherwise it may fail to improve the overall performance. It is easy to imagine a scenario of a badly designed interaction loop where channel monitoring parameters (see step 1 of figure 4) are flooding the network at the expense of useful traffic. Or even a badly orchestrated reconfiguration that results in some switches operating with different L1/L2 parameters (for example different carrier frequencies), thus making the nodes unreachable.

E. REMARKABLE ISSUES ABOUT SDN-SDR COEXISTENCE
The main lessons learned throughout the study of SDN-SDR coexistence in cross-layer architectures are as follows: • There are good technical and business reasons for researching and developing wireless systems in which SDN and SDR are hosted by the same node in an interactive mode. • Currently, there exists no standardized architecture that can combine SDR and SDN with the purpose of achieving a cross-layer optimal design. Some architectures have been presented in the literature as part of research projects and certain commonalities can be found among them, but more work is required in order to create a solid SDN-SDR coexistence framework. • The incorporation of SDR in SDN designs is expected to increase the traffic on the various control channels. Therefore, joint SDR-SDN approaches should carefully consider how to manage this communication overhead.  Further, the overall process of transmitting and using this information should be properly optimised.

V. CONCLUSION AND FUTURE DIRECTIONS
In this survey, we advocate that the joint consideration of SDN and SDR is a perspective whose time has come, for which there is much fertile ground in MANETs. First, the theoretical foundations of cross-layer design and optimization are presented, which form the backbone and motivation for such a cooperation. Next, we present several practical aspects of SDN and SDR, highlighting the various contri-butions derived from the literature as well as challenges, issues, and gaps that need to be addressed, including various controller-switch communication scenarios for SDN, SDN-SDR interaction and corresponding challenges.
Having this survey as a starting point, there exist several research threads that warrant further investigation, and we outline a few of them in the sequel.
Addressing bottlenecks of centralization. There are two main bottlenecks in applying centralized control and crosslayer optimization in MANETs that need further investigation in order to be addressed in an efficient manner: (i) The control signaling overhead and induced latency can erase the benefits of centralized optimization, especially when the CP communication is in-band. (ii) Centralized optimization algorithms can take too long to provide a good solution for them to be run in realtime, depending on the network size and optimization problem formulation.
For the first bottleneck, continuous research in advanced SDR architectures and algorithms can offer efficient dualband radios so that a single-hop, out-of-band control plane becomes an affordable option. At the same time, advanced centralized control algorithms have been proposed that can reduce the control overhead by using prediction mechanisms to calculate future-proof solutions. These algorithms proactively calculate and disseminate solutions under alternative future states, and/or achieve a good trade-off between centralized and distributed control, and they have shown great promise in mitigating the adverse effects of in-band control overhead.
Regarding the second bottleneck, there are numerous works trying to address different versions of the cross-layer optimization problem by developing sub-optimal algorithms which trade-off accuracy for computational time. However, the complexity of the full-fledged network configuration optimization problem (taking routing, scheduling, MAC, and PHY layer parameters into account) is so high that crosslayer optimization remains a very hard problem. At the same time, the objective function to be optimized varies according to the network operator objectives (throughput maximization, reduced latency, service delivery fairness, low energy consumption are only the most common ones). AI-driven cross-layer optimization. Cross-layer optimization has matured as a research area in the last three decades, and significant advances have been achieved, grounded primarily on methods that are based on mathematical models. These models have helped increase our understanding on factors in the control loop that affect performance and the control knobs in different layers that need to be jointly calibrated in order to achieve the defined optimization goals. The limited scope of model-driven approaches restricts them to controlling only the factors that can be explicitly modeled. However, in many cases, especially in large interconnected systems, such as networks, accurate models become hard to derive.
In the last few years however, an arising trend in the wireless network community is to migrate from modeldriven to data-driven approaches, by introducing in the control loop Artificial Intelligence (AI) and Machine-Learning (ML) methods which seek to take advantage of the massive amounts of measurement data that are generated at unprecedented rate. This trend is also evidenced by arising relevant workshops in major conferences. AI techniques drive research in Beyond 5G (B5G) wireless networking.
AI will have a significant role in reshaping next-generation networking and will help realize autonomous, intelligently operating wireless networks. Resource management, service deployment, policy control, performance monitoring, and performance prediction are expected to be benefited from AI-enabled techniques. Machine Learning (ML) techniques based on supervised learning have shown promising results, mainly on wireless propagation channel modeling and PHYbased control for infrastructure-based wireless networks. These ML models can be trained a priori and their parameters e.g., Deep Neural Network weights) can even reside on central nodes on which they are pre-loaded before operation. In order to facilitate the extension of AI models to layers higher than PHY in decentralized systems such as MANETs, the following questions need to be resolved: how to gather raw data, where to store them, how to train the ML models using data gathered at different nodes in the network, how to perform the computations on the data to infer actionable information, and where to store the trained ML models.
Online learning approaches in SDN. The centralized nature of SDN offers much fertile ground for applying ML and AI techniques. The SDN controller can gather network data and train ML models that can be used to drive its decisions, such as traffic routing. However, supervised learning approaches may be hard to realize since they rely on the availability of massive datasets, which is hard to assume, especially in certain scenarios, e.g., tactical ones. Consider for example the problem of deciding on the optimal controller placement, based on the criterion of minimizing the delays experienced between the controller and all switches. The following limitations exist. First, it is not easy to measure at any given time the delay between any pair of nodes so as to take decisions. This is because, through the SDN protocol, it is possible to measure only delays between the switches and the active controller. Second, even if such measurements were feasible through protocol modification, the information about delays at a time slot would not be very useful in the controller selection decision, since delays are likely to vary significantly (and sometimes, arbitrarily) in wireless settings.
In such scenarios, it is plausible to try to learn progressively a model in an online fashion, based on continuously received feedback. In the controller placement problem, such feedback would consist of the values of delay between a controller and the switches. It would be interesting to identify and study problem instances such as the one above for applying online learning techniques to SDN.
Application Interface design for SDR-SDN coexistence.
While the established theory and algorithms in cross-layer optimization provide valuable insights on how to efficiently run and control MANETs, there exist currently no standardized architecture and the associated protocols to put this vision into perspective. SDN can provide the bridge between centralized network control, that is inherent in SDN, and the distributed nature of mobile ad hoc networks. A properly designed SDN and SDR coexistence and interaction requires an appropriate API, optimized reconfiguration of PHY and MAC parameters at a local (node) level and orchestration at a network level in order to to deliver flexible, resilient, and ultimately better wireless network implementations.  His research interests include control, optimization and applied machine learning, with application in mobile computing, wireless networks, social networks, recommendation systems, smart energy grid, and cloud computing. He has co-authored over 160 papers in international journals and conferences. He