Maximizing the Probability of Message Delivery Over Ever-Changing Communication Scenarios in Tactical Networks

This letter introduces a stochastic model to maximize the probability of message delivery over ever-changing communication scenarios in tactical networks. Our model improves modern tactical systems implementing store-and-forward mechanisms organized in a hierarchy of layers for messages, IP packets and radios. The goal is to estimate the optimum redundancy for the user data-flow to overcome packet loss during changes in the link data rate, including disconnections. Experiments in a VHF network illustrate the numerical results from our model using messages with different sizes over two patterns of data rate change.


I. INTRODUCTION
C OMMUNICATION scenarios at the edge of tactical networks are exposed to several sources of randomness that can change the radio link data rate [1]. Therefore, tactical systems might have to deliver messages over ever-changing scenarios with both user data-flows and network conditions changing independently [2]. Given the wide range of military operations, it is challenging to design tactical systems that can thrive in arbitrary communication scenarios, also including link disconnections. Multi-layer control mechanisms, within tactical systems, rely on feedback from the radio and router to compute the current link quality (e.g., data rate, latency and packet loss [3]). The goal is to mitigate packet loss, doing flow control and adding redundancy, to increase the probability of message delivery over unreliable radio links.
We start with the hypothesis that the probability of message delivery can be computed and finally maximized using cross-layer information within a modern tactical system with interfaces to the radio, router, message queue and proxy/gateway. Our model assumes that the network conditions are quantized by a set of data rates updated within a finite time window. Thus, the probability of packet delivery is directly related to the amount of bits successfully transmitted during the time window. Assuming that the tactical system uses error correction techniques in different layers (e.g., [4]), the probability of message delivery can be described by a binomial distribution, where n is the number of packets including redundancy and p is the probability that a single packet is delivered. As a result, our model goes layer-by-layer computing probabilities to estimate the probability of message delivery.
Our hypothesis is verified in our test platform [5] using two exemplary patterns of link data rate change discussed in [1]. There, we quantified the inter-packet latency of three types of data-flow over a Very High Frequency (VHF) link changing in a given pattern. Here, we introduce a model to compute the probability of message delivery during those experiments. The goal is to calculate the close to optimal redundancy to increase the probability of message delivery over ever-changing network conditions including link disconnections. Therefore, the main contributions of this letter are the introduction of a stochastic model to maximize the probability of message delivery, its theoretical proof and the model's numerical output computed using results from experiments in a VHF network with real military radios.

A. Ever-Changing Communication Scenario
Modern tactical systems are organized into layers complementing each other through multi-layer control mechanisms to handle independent changes from both user data-flows (A) and network conditions (B), as illustrated in Fig. 1. This figure shows the end-to-end communication scenario with the sender and the receiver connected through a radio link, composing of an ever-changing communication scenario [1], [2], [6]. Each node has a control plane (c) and two chains: one for incoming (i) data-flows and another for outgoing (o) data-flows, both sitting in at least four layers, namely radio (0), packet (1), message (2) and proxy/broker (3). Notice the connection symmetry through a noisy channel, where the out chain from the sender is connected to the in chain of the receiver and vice-versa.
The sequence of messages from command and control systems (A) enter the system from layer 3 carrying a set of QoS requirements such as priority, reliability and time of expire (differentiated at layer 2), which are partially mapped to IP packets at layer 1. And the radio (layer 0) usually has a buffer with limited size that differentiates the packets by priority. Note that a multi-homed node with r radio networks will have r instances of this hierarchy of queues to handle the difference in both coverage and link data rate from military communication technologies.

B. Computing the Probability of Message Delivery
Let us assume that the network conditions in a communication scenario C is described as the five-tuple C = ( X 0 , B, θ, B T , T), where X 0 is the initial state vector, B is a set of matrices describing M different probability distributions, θ is the update function, B T is a set of transition matrices and T is the time distribution of the in-homogeneous Markov model, called Model B as we described in [1]. The stochastic model is composed of two nested Markov chains, one representing the link data rate changes and another one defining the distributions B of the pattern of data rate changes forming the communication scenario C. Given the initial state vector X 0 , we use this model to sample the sequence of data rate changes Σ describing the network conditions of the respective communication scenario C. Now, we are interested in the probability of a message being successfully transmitted from sender to receiver in Fig. 1, while the radio link is changing according to the distribution defined by C. For this purpose, our system implements an error correction technique called Reed Solomon Code ([4], [7]), meaning that if the message consists of k packets and n − k redundant packets, the message can be successfully delivered if at least k packets are delivered by sending a selective acknowledgement, which indicates the sequence numbers of the lost packets. If more than k packets are successfully delivered no re-transmission is required. Our goal is to avoid the overhead of re-transmissions by adjusting the system parameters to maximize the probability that at least k packets are transmitted.
Assuming independent errors that might cause packet loss, the probability for the receiver getting any 0 ≤ k ≤ n out of n sent packets is given by the binomial distribution as defined in (1). In this relation, p 0 is the probability that a single packet will be delivered and X is a random variable measuring the number of successfully transmitted packets [8]. The equation holds, because we have n k possibilities to sample a set of k packets out of Ω = {1, . . . , n} packets.
The probability that the random variable X is in state k is highly dependent on the current link state (link data rate) and thus dependent on the patterns P i sampled from Model B defined in [1]. As a result, if E 1 is the event X ≥ k , meaning that the message can be successfully delivered in the first round and E 2 is the event that the outer Markov chain of Model B is in state s(t) = P i , we get This construction can be extended to compute the probability p 2 of message delivery (at layer 2 in Fig. 1) during a communication scenario C consisting of I different patterns P 1 ∪ · · · ∪ P i ∪ · · · ∪ P I = C by finding an optimum configuration for In terms of k i and fulfilling the constraint I i=1 k i = k at the same time.

A. Probability of Packet Delivery
Let us sketch a solution for computing the probability of packet delivery p 0 (T w ) given a time window T w in seconds using the features collected during the communication scenarios generated in [1]. Thus, we assume that the data rate changes of the system follow an in-homogeneous Markov chain represented by are also Markov chains, each one representing the probability distribution of a specific pattern of data rate changes. Moreover, we assume that the distributions B 1 , . . . , B m together with the distribution λ are well known and therefore we have access to an oracle knowing the link states To describe arbitrary time windows associated to the time distribution T of communication scenario C, we define the , mapping a point in time t ∈ T to a time interval describing how many seconds the link (inner Markov chain) stays in state b(t) ∈ [0, . . . , 5]. Assuming that the probability of delivering a packet p 0 (T w ) is proportional to the amount of bits received within the time window T and the packet size is distributed by κ, the optimal data rate b opt (t) for almost sure delivery can be calculated using the ratio Using the relation in (5), we can compute the ratio between the current data rate b(t) and the optimal data rate b opt (t) for As a result from (4) and (5), we can use b ratio to introduce a function g(t, b(t), b opt (T w )) that computes an initial guess for the probability of packet delivery at layer 0 This function can be interpreted as follows. If g = 0, it means that the data rate b(t) is zero for the entire time window T w . Obviously no packets can be transmitted and also the probability of packet delivery p 0 (t) = 0. If g = 1, then the capacity of the link in the given time window T w is greater or equals the size κ of the packet and as a result p 0 (t) > 0.
) either for fixed |T w (t)| or b to adjust the system parameters s.t. g = 1 and as a result p 0 (t) > 0. Now, let us assume that g = 1 and b(t) > b opt (t) or more precisely b(t) = (1 + ) · b opt (t). This means that the link is able to handle an additional amount of data · b opt (t). This capacity can be used to increase the probability of packet delivery p 0 (t) by adding redundancy From these analysis of g, it follows that we can rewrite p 0 (T w ) as a conditional probability depending on the data rate b(t), the packet size κ and the time window T w p 0 (T w ) = P(X = 1|b(t), T w , κ).
To guarantee p 0 (t) > 0 even under ever-changing communication scenarios in a laboratory with military radios, we exploited the properties of function g by varying the parameters of g and as a result also of p 0 during a set of experiments over stable network conditions. Fig. 2 plots exemplary values of p 0 (T w ), for two sending rates 0.05 (left) and 2 (right) packets per second, as a function of the nominal data rates b(t) supported by our VHF radios (.6, 1.2, 2.4, 4.8 and 9.6 kbps), the packet size κ (1.3 kB) and the time window T w (t) (from 0 to 300 sec). These results over stable conditions are used as ground truth to compute p 0 and also p 2 for two communication scenarios including link disconnections in the following sections.

B. Maximizing the Probability of Message Delivery
Now, we can use the probability p 0 (T w ) and the properties of g to maximize the probability of message delivery in everchanging communication scenarios by adding redundancy. For this purpose, we assume that we are given the communication scenario C, a message consisting of k data packets and the distribution of the packet size κ. Furthermore, we consider that we can access the probabilities p 0 (T w ) = P(X = 1|b(t), T w (t), κ) for different time windows T w with maximum size t wmax as shown in Fig. 2. Referring to the problem in Section II-B, we state an optimization problem to find the maximum value of p 2 given C: Problem 1 (Maximizing the Probability of Message Delivery With a Minimum Amount of Redundancy): Let F X |C (n = r + k ; p 0 ; k ) be the objective function describing the binomial distribution, in (3), of successful message delivery given communication scenario C and the number of data packets k, find the optimum amount of redundancy r s.t.
Here, the patterns P i ⊆ Σ of link data rates are generated by the matrices B m ∈ B defining the communication scenario C. The parameters k i represent the number of packets that should be transmitted, assuming that the link states change according to pattern P i . Since we are interested in sending a message that consists of k packets, the parameters k i must sum up to k. The idea is to find an optimal amount of redundancy r to send at least X = k data packets during a scenario composed of different patterns P i meaning that r maximizes P(X ≥ k i |P i ) by fulfilling the constraint X = m i=1 k i ≥ k . Algorithm 1 solves this problem (8) sampling the sequence of data rate changes Σ and the patterns P i describing the communication scenario C. Then we compute the probability of transmitting k i data packets for each possible combination of time window t w ≤ t wmax , pattern P i and number of data packets k i ≤ k . The maximum optimum of redundancy r i that can be added to the system is computed as referring to Section III-A. The corresponding probabilities are stored in the data structure S k and added to S pt afterwards. In sequence, the algorithm computes all possible configurations maximizing I i=1 (P(X ≥ k i |P i )), s.t. I i=1 k i ≥ k and each pattern P i is considered only once. We keep the configuration that uses a minimum amount of redundancy r min = I i=1 r i = n i − k i . The following theorem shows that for any arbitrary communication scenario generated by Model B , there exists an algorithm that finds an optimum solution for the problem defined in (8).
Theorem 1 (Optimized Message Delivery in Ever-Changing Communication Scenarios): Given a non-empty, ever-changing Algorithm 1 MAXPROB Input: C = ( X 0 , B, θ, B T , T, λ), P 0 , Msg, κ, tw max , seed Output: Optimum amount of redundancy r, maximized probability P(X ≥ k |C ) Initialization : 7: for pattern P i ⊆ Σ do 8: for k i in 0, . . . , k do 9: Given k i packets, compute the maximum amount of redundancy r i = t 2 t=t 1 · b opt (t) that the system can handle during Compute the probability P(X ≥ k i |P i ) with redundancy r i = n i − k i = r i /κ by using p 0 ∈ P 0 11:  tw + + 21: end while 22: return (p best , r best ) communication scenario C = ( X 0 , B, θ, B T , T), the probabilities P 0 for different end-to-end delays 1, . . . , t wmax over stable system conditions, the number k > 0 of data packets defining the length of a message Msg, the distribution of the packet size κ and the maximum end-to-end delay t wmax , MAXPROB (Alg. 1) computes an optimum solution to the problem in (8).
Proof: Let C, k, κ and t wmax be as defined in the statement of the theorem. First we show that Alg. 1 finds the maximum probability for delivering at least k packets by using a minimum amount of redundancy or in other words Alg. 1 finds an optimum solution to problem in (8). The idea is to proof inductively that for each possible end-to-end delay 1 ≤ t w ≤ t wmax , p best is always maximized by the optimal amount of redundancy r best . Initially, we sample the sequence of states Σ consisting of patterns P i , i ∈ I using the Sample algorithm from [1]. For the correctness of this algorithm we refer to our previous investigation. Moreover, the data structures S pt ,S k storing the probabilities for each pattern P i and packet size k are empty and the variable k is set to the number of data packets of the Msg (lines 2-5).
Let t w = 1, then lines 7-12 compute the maximum redundancy r i = n i − k i = r i /κ that the system can handle and the corresponding probability P(X ≥ k i |P i ) by using the probabilities p 0 ∈ P 0 for each possible combination of a pattern P i and number of packets k i ≤ k . The probabilities for fixed pattern P i are stored in the data structure S k and added to S pt afterwards. As a result, after line 13, S pt stores all probabilities for delivering k packets by considering each possible pattern P i ⊆ Σ. In sequence (line 15), the algorithm computes all possible configurations maximizing I i=1 (P(X ≥ k i |P i )), s.t. I i=1 k i ≥ k and each pattern P i is considered only once. We keep the configuration that uses a minimum amount of redundancy r min = I i=1 r i = n i − k i . Since p max > p best is obviously true, we update p best and r best . By line 15 (r best ) is an optimal configuration to problem (8) for t w = 1 with probability.
The induction step (t w ) → (t w + 1) can be done along the same lines. Since p best and r best are only updated if the if the algorithm finds a higher probability p max , the tuple r best is always an optimal solution with probability p best . As a result Alg. 1 is correct and outputs the maximum probability for delivering at least k packets by adding a minimized amount of redundancy that does not exceed the capacity of the system. It remains to show that the algorithm terminates for each possible configuration of C, k, κ and t wmax . But since C is always finite and k, κ and t wmax this is trivial.

C. Multi-Layer Stochastic Uncertainty Model
Given the probabilities p i (T w ), i ∈ Layers[0, 2] and the distribution of the packet size κ, finally we introduce a two layered stochastic uncertainty model over time T. Referring to [9], we are modeling the tactical system uncertainty and, since the probability p i (T w ) depends on the data rates b(t) generated by Model B , the stochastic uncertainty model can be defined in a similar way. To this end, we assume that M is an instance of Model B and that we are given the 5-tuple C = ( X 0 , B, θ, B T , T) required to sample the sequence Σ by computing φ sample as proposed in [1]. Now we can define two functions to convert the instance M to an instance U of an stochastic model describing the uncertainty of the underlying tactical system by replacing the system states s in (t) ∈ [0, . . . , 5] (radio link data rates) of the inner Markov chain of M by probabilities p 0 and the pattern P j representing the states of the outer Markov chain by p 2 .
To this end, we define the function ψ in mapping each system state s in (t) to a probability p 0 as Moreover, we can fix the state of the outer Markov chain for a time interval [t 1 , t 2 ] with 0 ≤ t 1 < t 2 ≤ T , t 1 , t 2 ∈ T, meaning that the system follows a pattern P j = [s in (t 1 ), . . . , s in (t 2 )] generated by a single matrix B j ∈ B. Given the size k of a message, we can now replace P j by the maximized probability p 2 using ψ out defined as As a result, this model describes the distribution of uncertainty for packet delivery in the lower layer 0 and the probability of message delivery in the higher layer 2.

IV. RESULTS
In this section, we discuss experimental results observed in the end-to-end communication scenarios earlier shown in  Fig. 1. We reused two loop patterns, L 1 and L 2 , and the VHF network from the experimental setup described in [1]. Both patterns of change include very low link data rates (i.e., 0.6 and 1.2 kbps) and even link disconnections (0 kbps) resulting in packet loss. Here, we demonstrate how redundancy can improve the message/packet delivery over the loop patterns L 1 (5% loss) and L 2 (54% loss), both with 200 states updated every 10 seconds. We have chosen these two patterns because they concentrate the disconnections in the middle of the experiment. The messages were composed by 1, 3, 9, 18 and 54 packets, and the sender packet rate was distributed according to (0.05, 0.075, 0.1, 0.2, 0.5, 1, 2) packet/second.
The goal is to illustrate the numerical output of our model discussed earlier in Section III. Table I lists the results for 0.2 packet/second, and four levels of redundancy 0, 100, 200 and 500%. In this table, L * 1 and L * 2 are the baseline results computed supposing that p 0 is known for these patterns of change. In contrast, all other results for L 1 and L 2 are computed using p 0 from the nominal data rates supported by our VHF radios, as plotted earlier in Fig. 2. Using the results from these experiments, we computed the probability of message delivery for different maximum end-to-end delay |T w | ∈ {1, . . . , 300} using Alg. 1.
Notice that without redundancy, r(t) = 0%, the probability of message delivery is about 60% for messages with 1 packet and goes decreasing until 0% for messages with 18 or more packets. This fact motivated this investigation and is visible in the first four lines of Table I, listing the probabilities measured (L * 1 and L * 2 ) and the probabilities computed using Alg. 1 (lines 3 and 4 in the table). Moreover, we observed that adding redundancy can significantly increase the probability of message delivery for both loop scenarios. For messages consisting of a single packet, we need up to 500% of redundancy to guarantee reliable message delivery. For messages consisting of 3 or more packets, we see that even 100% redundancy can lead to satisfying results (i.e., probabilities close to 1). Fig. 3 complements the results from this table by showing the probability for message delivery as a function of different time windows T w for six messages sizes, namely 1, 3, 9, 18 and 54 packets per message. In this figure, there are two examples representing 100% (left) and 200% (right) for L 1 showing that the probabilities converge with respect to the size of the maximum end-to-end delay |T w |; also called time window. This is an important result, because it yields to the assumption that we can replace the parameter t wmax by another parameter describing the convergence rate of Alg. 1.

V. CONCLUSION
This letter introduced a stochastic model to maximize the probability of message/packet delivery using the hierarchy of layers from modern tactical systems. The goal was to estimate the optimum redundancy level to mitigate packet loss in communication scenarios with link disconnection therefore increasing the probability of delivering messages. Thus, we started with the hypothesis that transport protocols can use our model to proactively add redundancy to reduce the packet loss observed during radio link disconnections. Our hypothesis was verified with experiments sending messages with different sizes through a VHF link with data rate changing in two different patterns. The experimental results suggest that our stochastic model can compute close to optimal parameters for a transport protocol using redundancy to overcome packet loss during link disconnections, also avoiding data overhead from packet acknowledgements and packet re-transmissions.