• Abstract

# Lump-Sum Markets for Air Traffic Flow Control With Competitive Airlines

When weather disrupts air traffic, allowing airlines to competitively bid for priority treatment increases their satisfaction with results and may aid overall system efficiency.

Air traffic flow control during adverse weather conditions is managed by the Federal Aviation Administration in today's air traffic system, although it is the individual airlines that are in the best position to assess the costs of disruptions to scheduled operations. To improve the efficiency of resource allocation, a market mechanism is proposed that enables airlines to participate directly in the flow control decision-making process. Since airlines can be expected to behave strategically, a lump-sum market mechanism is used for which existence of a Nash equilibrium and a bound on the worst case efficiency loss have been shown for agents that anticipate the effects of their own bids on resource prices. The convergence properties of this mechanism are studied for a two-player game with linear utilities, which reveals that restricting the airline bid update step-size can result in a wider range of stable bidding processes. The mechanism is then applied to an air traffic flow control scenario for multiple airports in the northeastern United States, which demonstrates the feasibility of performing market-based resource allocation within the time horizon for reliable weather predictions.

SECTION I

## INTRODUCTION

Air traffic management for the U.S. National Airspace System (NAS) is one of the essential roles undertaken by the Federal Aviation Administration (FAA) in maintaining safe commercial flight operations. The air traffic control system is responsible for ensuring that resource limits such as airport takeoff and landing rates are not exceeded, as well as ensuring that minimum en route aircraft separation is maintained for up to 45 000 flights a day [1]. Typical operations, as depicted in Fig. 1, can require simultaneous management of upwards of 6000 aircraft. This complex task is aggravated by adverse weather conditions caused by inclement weather, which are difficult to predict beyond three hours in advance, and complicated by the many stakeholders for whom disruptions result in significant financial burdens. Delays incurred by any individual flight can result in repercussions throughout the network, as aircraft, crews, and passengers may all be scheduled to continue on subsequent flights. The result is a highly connected, dynamic, and unpredictable resource allocation problem with significant financial ramifications for the airlines.

Fig. 1. Air traffic visualization for the U.S. National Airspace. (Courtesy of NASA Ames and FACET [2].)

The FAA has three main avenues for reconfiguring scheduled operations in the face of airspace disruptions: ground delay programs (GDPs), miles-in-trail restrictions, and flight reroutes. GDPs can be used to delay network traffic bound for specific constrained resources, such as destination airports and en route airspace. Miles-in-trail restrictions specify en route spacing on crowded airways to smooth flow into intersections and congested areas. Flight reroutes ensure aircraft are redirected around weather disruptions and are managed through a national playbook of fixed procedures developed through years of experience. Currently, these methods address multiple stresses to the system individually. NAS-wide coordination of these actions is managed manually at the FAA Air Traffic Control System Command Center (ATCSCC), with verbal input from members of the Collaborative Decision Making (CDM) initiative,1 which currently enables a common situational awareness as well as airline participation in GDPs. The CDM initiative is especially important as a model program because it is the first for which incentives were designed to enhance airline participation [3], since rationing resources by schedule eliminates the penalty formerly faced by airlines when providing accurate information about flight changes.

As demand for resources increases, however, the need for the ATCSCC to provide efficient coordinated national flow control decisions will only increase, and the research community has therefore proposed a variety of air traffic modeling and control techniques that enable NAS-level planning in real time [4], [5], [6], [7]. By aggregating individual flight plans into flows over a fixed network and defining capacity limits in both en route and terminal areas in terms of flow volumes, significant reductions in computational complexity are achieved over more detailed models that manage all flights individually. Though the ability to synthesize trajectories for each aircraft is lost, appropriate traffic levels for sectors are computed to meet regional demands due to congestion and weather. This approach is in line with the hierarchical structure of the NAS, in which the discrete problem of aircraft separation is handled by sector controllers, while air traffic management at the center level often deals primarily with regional flow concerns. Another issue arises due to the fact that the costs incurred when flights must be delayed are primarily borne by the airlines, which are in the best position to assess the relative value of favoring one set of flights over another. Recent advances have focused on long-term allocation of airport slot resources [8], [9], or specifically on the GDP process [10], but do not consider disruptions to daily operations due to weather. To achieve efficient utilization of airspace resources, up-to-date private airline cost information must be included in the traffic-flow decision-making process, a need that is complicated by the competitive nature of airline operations.

In many engineering applications, such as Internet routing and electricity generation, market mechanisms have been proposed as a way to incorporate preference information into resource allocation decisions with competing agents [11], [12]. Market mechanisms assign an explicit price to each resource, and agents balance the cost of purchasing resources with the value of those resources to them [13]. At each step in the process, the agents seek to minimize their costs by purchasing resources at market prices, after which the central coordinator updates resource prices in order to penalize excess demand. In previous work [7], a network flow model and market mechanism were presented for the air traffic control system, allocating airspace resources on a three-hour time horizon. The resources include take off, en route and landing traffic flows, and are allocated based on airline bids and centrally defined prices. Such a system could be implemented as a natural extension to the existing Collaborative Decision Making communications framework by requiring airlines to submit bids for traffic flow in areas affected by capacity restrictions due to weather.

Analysis of market mechanisms often relies on the assumption that agents treat prices as exogenous. For large numbers of agents with limited market share, such an assumption is reasonable. Unfortunately, applications with a small number of agents that command significant fractions of the available resources, such as the air traffic control problem, are susceptible to price manipulation by the agents. It becomes implausible to accept the assumption that agents would act without knowledge of how their actions affect market prices. Instead, agents must be thought of as price-anticipating, which is to say, capable of predicting the effect of changes in their own requests on the market. This can lead to arbitrary efficiency loss for certain market mechanisms [14], [15] and requires the definition of an alternate mechanism for which such a negative result can be avoided.

One such mechanism exists in the form of the lump-sum market, which was recently studied in the context of Internet congestion control [16], [17]. Instead of having agents request resource allocations directly, the lump-sum market, as its name suggests, requires agents to specify lump-sum payments they are willing to make for each resource, resulting in an elastic demand that can always be satisfied by adjusting prices relative to the cumulative payment per resource. Existence and uniqueness results for a Nash equilibrium have been established for price-anticipating agents in the single resource case [18], and existence of Nash equilibria in the case of multiple network resources [17]. What makes this market mechanism appealing is that a proportional bound on efficiency loss relative to the central solution has also been demonstrated for flow networks [17], limiting the possible losses that occur if agents act as price-anticipators.

The convergence properties of the lump-sum market are not well understood, but this paper presents one set of preliminary results in this area. The two-player game for agents with linear utilities is considered in detail, and convergence properties are presented for continuous steepest ascent dynamics and for discrete best response dynamics. This investigation reveals marked differences between the convergence properties of the continuous and discrete dynamics, a fact that is not immediately obvious, as the discrete dynamics are not merely defined as discretized versions of the continuous dynamic (for which it is well understood that step size adversely affects convergence). Rather, the discrete dynamic is defined as a direct update to an agent's optimal bid, or best response, if all other agents are held at their current bids. As the difference in the marginal utility of agents increases, convergence of this dynamic fails due to the fact that small changes in one bid result in large changes in the other agent's best response.

For the air traffic control problem, these results present a method for improving the convergence of lump-sum markets with multiple airlines, arbitrary convex cost functions, and numerous resources. If iterative best response bidding shows oscillatory pricing behavior in a specific situation, it is likely that fixing a maximum bid change between rounds will improve convergence, by mimicking the effect of requiring continuous dynamic updates. In practice, iterative best response bidding between price-anticipating airlines was observed to converge quite quickly, as is demonstrated in a flow control scenario presented in Section V. With eight airports, four airlines, and a 3-h time horizon, the scenario is indicative of the problem size that can be solved in real time on a standard desktop computer. The mechanism can also be scaled to manage larger networks if distributed computation platforms are considered for each of the airlines.

Market mechanisms exhibit known deficiencies when implemented for engineering applications. The mechanisms do not consider the effect of repeated daily operation of the market on the airline strategies, nor do they consider the possibility that airlines might take into account the multiple stages of the bidding process. It will be necessary to ensure that price fluctuations do not result in unsafe situations, although this concern is mitigated by the 3-h planning horizon. There are also issues relating to the adoption of a novel resource allocation method which represents a paradigm shift in U.S. air traffic management, as most en route resources are currently free of charge and airport fees are not slot dependent. Despite these open issues, this paper presents a first step toward obtaining cost information from strategic airlines and incorporating it into air traffic control decision-making. Resolving these deficiencies for resource allocation in air traffic control remains an area of future work.

This paper proceeds as follows. Section II defines a single resource lump-sum market and restates the main known results concerning existence and efficiency of a Nash equilibrium. The convergence properties of the lump-sum market for two agents with linear utilities are then investigated in Section III, for both iterative best response play and continuous steepest ascent dynamics. In Section IV, the air traffic network flow resource allocation problem is restated for completeness, as first defined in [7], and the lump-sum mechanism definition is extended to the network flow problem, as described in [17]. Simulation results are presented for an air traffic network consisting of eight airports in the northeastern United States over a 3-h horizon with inclement weather affecting the destination airport in New York. Finally, some of the difficulties inherent in implementing the proposed mechanism and areas of future work are described in Section VI.

SECTION II

## LUMP-SUM MARKET MECHANISM

### A. Lump-Sum Market for a Single Resource

Consider a system of a single resource with constrained quantity . Competing for the resource are agents who value their allocation according to a utility function . The agent utility functions are assumed to be both increasing and concave over the compact space defined by Xj = [0,C]. A centralized formulation of the efficient allocation problem is defined in Problem P2.1. TeX Source \matrix{\noalign{\hrule\vskip 2.5pt}\smash{\vrule depth 43.5pt height 42pt}&\eqalign{&\hbox{ Central Allocation Problem}\cr\noalign{\vskip 3pt} &{\hskip 20pt}\hbox{maximize}\quad{\hskip 1pt}\sum\limits_{j\in{\cal J}}\mathtilde{U}_{j}(x_{j})\cr\noalign{\vskip 4pt}&{\hskip 20pt}\hbox{subject to}\quad\sum\limits_{j\in{\cal J}}x_{j}\leq C}\qquad{\hbox{(P2.1)}}&\smash{\vrule depth 43.5pt height 42pt}\cr\noalign{\vskip 9pt\hrule}}This convex program can readily be solved if all utility information is known, but, for systems with independent agents and a competitive environment, it is improbable that this information will be available to a central authority. Instead, market mechanisms can be employed to reach an agreement on resource allocation amongst agents without requiring full disclosure of utility functions.

A lump-sum market is defined in which each agent specifies a lump-sum payment for the resource. This results in an elastic demand that balances demand with capacity using the pricing rule TeX Source $$\lambda={\sum w_{j}\over C}.\eqno{\hbox{(1)}}$$The resource price is defined as and the notation ∑ wj is used when summing over all agents. An agent's lump-sum bid results in the allocation TeX Source $$x_{j}={w_{j}\over\sum w_{j}}C.\eqno{\hbox{(2)}}$$

In abstracting resource markets for analysis, it is essential to explicitly define models of agent information and behavior that accurately capture the likely actions of agents in the real system. For markets with large numbers of similarly sized participants, it is reasonable to assume that agents assume prices are beyond their influence. Such agents are said to be price-taking and operate rationally with respect to the fact that they believe their bids have no effect on the market prices they experience. Restricting the price to be strictly positive (i.e., requiring that the resource is contested), a price-taking agent in a lump-sum market optimizes the following local utility TeX Source $$U_{j}^{PT}(w_{j};\lambda)=U_{j}\left({w_{j}\over\lambda}\right)-w_{j}.\eqno{\hbox{(3)}}$$The notation for the arguments of the utility function UjPT(wj;λ) employs a semicolon to distinguish between arguments that are controlled by agent j, such as its bid, and those that are assumed by agent j to be determined exogenously, such as the price.

For markets with only a few participants, however, it is unreasonable to assume such naïvety on the part of competitive agents. Since the pricing rule for lump-sum markets can be assumed to be publicly available, it is likely that agents will anticipate the effect of updating their bids on market prices. This behavior is referred to as price-anticipating. A modified price-anticipating agent utility function is defined that incorporates the resource pricing rule and is expanded to account for the case in which all bids are zero TeX Source $$U_{j}^{\rm PA}(w_{j};w_{-j})=\cases{U_{j}\left({w_{j}\over\sum w_{j}}C\right)-w_{j},&if w_{j}>0\cr U_{j}(0),&otherwise.}\eqno{\hbox{(4)}}$$For notational simplicity, the term wj refers to the set of all bids other than agent j's.

The appropriate equilibrium condition for a market with price-anticipating agents is that of the Nash equilibrium, where each agent prefers its current bid to any other, given that all other agents hold their bids fixed. A Nash equilibrium is defined as a vector of bids wNE≥ 0 such that TeX Source $$U_{j}^{\rm PA}\left(w_{j}^{\rm NE};w_{-j}^{\rm NE}\right)\geq U_{j}^{\rm PA}\left(w_{j};w_{-j}^{\rm NE}\right),\quad\forall w_{j}\in\BBR_{+}.\eqno{\hbox{(5)}}$$

### B. Equilibrium Properties for a Single Resource

Two key results in the analysis of lump-sum markets with price-anticipating agents are restated here, which establish existence and uniqueness of the Nash equilibrium as well as a bounded efficiency loss relative to the optimal solution to the central allocation problem of no more than 25%.

The following theorem shows that a solution to the game exists in the form of a unique Nash equilibrium.

#### Theorem 1 (Existence, Uniqueness [18])

Assume that for J > 1, each agent j has a concave, increasing, and continuously differentiable utility function Uj. Then there exists a unique Nash equilibrium wNE≥ 0 that is the unique solution to the modified central allocation problem TeX Source \matrix{\noalign{\hrule\vskip 2.5pt}\smash{\vrule depth 43.5pt height 42pt}&\eqalign{&\hbox{Modified Central Allocation Problem}\cr\noalign{\vskip 3pt} &{\hskip 20pt}\hbox{maximize}\quad{\hskip 1pt}\sum\limits_{j\in{\cal J}}\mathtilde{U}_{j}(x_{j})\cr\noalign{\vskip 4pt}&{\hskip 20pt}\hbox{subject to}\quad\sum\limits_{j\in{\cal J}}x_{j}\leq C}\qquad{\hbox{(P2.2)}}&\smash{\vrule depth 43.5pt height 42pt}\cr\noalign{\vskip 9pt\hrule}}where TeX Source $$\kern-12pt\mathtilde{U}_{j}(x_{j})\!=\!\left(1-{x_{j}\over C}\right)U_{j}(x_{j})+{x_{j}\over C}\!\left({1\over x_{j}}\int\limits_{0}^{x_{j}}\!U_{j}(y)dy\!\right)\!,\ \ \forall j\in{\cal J}\eqno{\hbox{(6)}}$$and the allocation is defined by (2).

Even though the modified utility function does not have a clear interpretation, it is a useful construct in that it enables the proof of existence and uniqueness of the Nash equilibrium. The following theorem ensures that efficiency loss in the game is bounded.

#### Theorem 2 (Efficiency Loss [17])

Let xS be the unique solution to the central allocation problem and xG be the unique solution to the game (U1PA,…,UJPA) and hence to the central allocation game. Then TeX Source $$\sum_{j\in{\cal J}}U_{j}\left(x_{j}^{G}\right)\geq{3\over 4}\sum_{j\in{\cal J}}U_{j}\left(x_{j}^{S}\right).\eqno{\hbox{(7)}}$$Furthermore, for any set of agents J, there exist linear utilities for which this bound is tight.

This result is the main reason that lump-sum markets have recently been proposed as an alternate to resource markets for network resource allocation problems [17].

SECTION III

## CONVERGENCE PROPERTIES OF LUMP-SUM MARKETS

In order for a mechanism such as the lump-sum market to be useful in an engineering application such as air traffic control, its convergence properties must be understood. Convergence results for best response dynamics are typically difficult to achieve, although some positive results have been established [19], [20]. By focusing on the two-player game with linear utilities, it is possible to gain some insight into the convergence properties of the lump-sum mechanism in general. To this end, the utility function for each price-anticipating agent is defined as TeX Source $$U_{j}^{\rm PA}(w_{j};w_{-j})=c_{j}{w_{j}\over\sum w_{j}}C-w_{j},\quad j=\{1,2\}$$where cj is agent j's constant marginal rate of return for additional units of the resource. Throughout this section, the symmetry of the problem is exploited to simplify notation by assuming that c2/c1≥ 1, without loss of generality.

### A. Best Response Functions

The best response functions for each agent can be analytically computed for linear utilities by solving the first-order necessary conditions for optimality of agent j's optimization problem TeX Source $${\partial\over\partial w_{j}}U_{j}^{\rm PA}={c_{j}\sum w_{-j}\over\left(\sum w_{j}\right)^{2}}-1=0,\quad\sum w_{-j}\ >\ 0,\eqno{\hbox{(8)}}$$where the summation ∑ wj is over all bids other than agent j. Let . Solving (8) for wj yields the agent's best response function TeX Source $$\beta_{j}(w_{-j})=\max\left(0,\sqrt{c_{j}\sum w_{-j}}-\sum w_{-j}\right) \eqno{\hbox{(9)}}$$which is defined for all values of ∑ wj > 0. If the sum of the other agents' bids is zero, the best response function is undefined, as any positive bid will capture all available resources. Fig. 2 presents the best response functions for two agents.

Fig. 2. Best response functions for two agents with utility slopes c1 = 1 and c2 = 2. Solid curves are best response functions and dashed lines indicate maximum bids, with agent 1 in green (light gray) and agent 2 in red (dark gray). The equilibrium is denoted w∗.

### B. Nash Equilibrium

At the Nash equilibrium of the game, denoted w∗, both agents bid their best responses: w1 = β1(w2∗) and w2 = β2(w1∗). The equilibrium can be found by taking the composition mapping β12(w1)) and setting it equal to w1 to obtain TeX Source \eqalignno{w_{1}=&\,\beta_{1}\left(\beta_{2}(w_{1})\right),\cr=&\,\sqrt{c_{1}(\sqrt{c_{2}w_{1}}-w_{1})}-(\sqrt{c_{2}w_{1}}-w_{1})&\hbox{(10)}}which results in TeX Source $$c_{1}^{2}(c_{2}w_{1})=(c_{1}+c_{2})^{2}w_{1}^{2}.\eqno{\hbox{(11)}}$$The equilibrium of the system occurs at TeX Source $$\left(w_{1}^{\ast},w_{2}^{\ast}\right)=\left({c_{1}^{2}c_{2}\over(c_{1}+c_{2})^{2}},{c_{1}c_{2}^{2}\over(c_{1}+c_{2})^{2}}\right).\eqno{\hbox{(12)}}$$

### C. Discrete Dynamics

The discrete dynamic can now be defined by setting the order in which agents respond to changes in each others' bids, be it sequentially, concurrently, or randomly. Let k index the time step; then the concurrent discrete dynamic w(k+1) = βc(w(k)) is defined for each agent as TeX Source $$w_{j}(k+1)=\beta_{j}^{c}\left(w(k)\right)=\beta_{j}\left(w_{-j}(k)\right),\quad\forall j\in{\cal J}.\eqno{\hbox{(13)}}$$The sequential discrete dynamic w(k+1) = βs(w(k)) is defined as TeX Source \eqalignno{w_{j}(k+1)=&\,\beta_{j}^{s}\left(w(k)\right)\cr=&\,\cases{\beta_{j}\left(w_{-j}(k)\right),&if k+1\ {\hbox{mod}}\ j=0\cr w_{j}(k),&otherwise.}&\hbox{(14)}}

Examples of the sequential update for the two-player game can be seen in Fig. 3 for varying values of agent utilities. Note that as Agent 2's utility per unit resource increases relative to Agent 1's, the sequential update dynamics become less stable, resulting in limit cycle behavior in Fig. 3(c) and instability in Fig. 3(d).

Fig. 3. Trajectories of sequential discrete dynamics for utility slope ratios c2/c1 = 1, 4, 6, 8. Best response curves are solid lines, discrete trajectories are dashed lines, the initial bids are denoted by a square, and the equilibrium is denoted by a diamond. (a) c2/c1 = 1; (b) c2/c1 = 4; (c) c2/c1 = 6; and (d) c2/c1 = 8.

#### 1) Local Stability

Local stability of the equilibrium can be determined by investigating the eigenvalues of the linearized dynamics by the Hartman–Grobman theorem, stated here in its original continuous form.

#### Theorem 3 (Hartman–Grobman [21])

If the Jacobian linearization Df(w) of the system at the equilibrium w∗ has no zero or purely imaginary eigenvalues, then there exists a neighborhood Un and a homeomorphism h: Un that maps trajectories from the system to its linearization. In particular, h(w∗) = 0, and the homeomorphism can be chosen to preserve the parameterization by time.

As a result, stability of the equilibrium with linearized dynamics applies to the nonlinear dynamics over some local region about the equilibrium. If the linearized dynamics have eigenvalues in the left half-plane, the nonlinear dynamics are locally exponentially stable for an open neighborhood of the equilibrium. Similarly, if any eigenvalue of the linearized dynamics is in the right half-plane, the nonlinear equilibrium is unstable. An analogous theorem exists for the discrete case [21], and local stability can be guaranteed by ensuring eigenvalues of the linearized discrete system are located inside the unit circle.

Starting with the concurrent discrete dynamic, the linearized dynamics about w∗ are TeX Source \eqalignno{\left(w(k+1)-w^{\ast}\right)=&\,D\beta^{c}(w)\vert_{w^{\ast}}\left(w(k)-w^{\ast}\right)\cr=&\,\left[\matrix{0&\displaystyle{c_{1}-c_{2}\over 2c_{2}}\cr\displaystyle{c_{2}-c_{1}\over 2c_{1}}&0}\right]\left(w(k)-w^{\ast}\right)&\hbox{(15)}}with eigenvalues TeX Source $$\lambda=\left[\matrix{\displaystyle{c_{1}-c_{2}\over 2\sqrt{-c_{2}c_{1}}}\cr\cr\displaystyle{c_{2}-c_{1}\over 2\sqrt{-c_{2}c_{1}}}}\right]\eqno{\hbox{(16)}}$$which remain inside the unit circle for ratios of . Above this ratio, the equilibrium is locally unstable. Note that the sequential update procedure results in the same linearization as the concurrent update, as can be demonstrated by considering the composition of two sequential updates.

#### 3) Convergence

Depending on the ratio of utility slopes c2/c1, the discrete dynamics evolve in a number of manners. This section first identifies the ratios for which convergence is possible, guarantees convergence for a subset of these ratios, and then describes the behavior that results above these ratios.

The maximum bid of either agent is found by maximizing its best response function over all bids by the other agent. Setting the best response function derivative to zero for agent 1 yields TeX Source \eqalignno{{\partial\beta_{1}\over\partial w_{2}}=&\,{\sqrt{c_{1}}\over 2\sqrt{w_{2}}}-1=0\cr w_{2}^{\max}=&\,c_{1}/4\cr\beta_{1}\left(w_{2}^{\max}\right)=&\,c_{1}/4.&\hbox{(17)}}The maximum bid by agent 1 occurs when agent 2 bids c1/4 and is also c1/4. Convergence to the equilibrium is shown for all feasible initial bids w0∊Λ = (0,c1/4]× (0,c2/4] for a limited range of utility slope ratios.

#### Theorem 4

Given utility slope ratios of c2/c1∊[1,9/4), dynamics defined by (14), and any initial bid pair w0∊Λ, the system converges to the equilibrium TeX Source $$w^{\ast}=\left({c_{1}^{2}c_{2}\over(c_{1}+c_{2})^{2}},{c_{1}c_{2}^{2}\over(c_{1}+c_{2})^{2}}\right).$$Please refer to the Appendix for the details of the proof.

For ratios of , it is possible to demonstrate that the best response function of agent 2, β2(w1), is positive for all w1∊(0,w1max]. This ensures that the only games in which either agent will post a bid value of zero occur when agent 2 bids above the maximum value at which agent 1 will participate. Setting β1(w2) = 0 yields w1pmax = c1 as that maximum value, and then setting β2(w1max) = w1p max and solving for the ratio c2/c1, gives a limit of c2/c1 = 25/4. Therefore, for all c2/c1 > 25/4, the discrete dynamic outcome may be undefined, and for all , the system is not able to converge to the equilibrium, as it is locally unstable, nor is it able to exit the region of feasible bids (0,c1/4]× (0,c2/4]. Instead, all trajectories must continue throughout the feasible region as t goes to infinity.

In summary, the two-player game with discrete dynamics can be subdivided into four types based on the ratio of utility slopes c2/c1.

• For ratios of , the equilibrium is locally stable, and convergence to the equilibrium can be guaranteed for all initial values of w∈(0,c1]×(0,c2] and utility slopes c2/c1 < 9/4.

• For slopes , no guarantee is provided, although it is our conjecture that all trajectories of the discrete dynamics do converge to the equilibrium.

• If the ratio of , two further possibilities exist. First, for ratios of c1 < 25/4, convergence to the equilibrium is not possible and a limit cycle occurs.

• Then, for ratios c2/c1 > 25/4, the possibility arises that one agent is forced to bid zero, at which point the best response function for the other agent is undefined, and the outcome is undefined.

Returning to Fig. 3(a)–(d), it can be seen that in scenarios (a) (b), the discrete dynamic achieves the equilibrium; in (c), a limit cycle results, and in (d), a zero bid is placed by agent 1, resulting in an undefined outcome. As expected, the ratios of c2/c1 for which each of these behaviors occurs fall into the regions described in this section.

This result is interesting from the point of view of applying the mechanism to the air traffic flow control problem, as it demonstrates that even with only two price-anticipating airlines, it is possible that iterative best response bidding may not converge to an equilibrium, leaving controllers without a traffic-flow solution to implement. The following section performs a similar analysis for a continuous airline bidding update model with more favorable results.

### D. Continuous Dynamics

Instead of allowing an agent to bid its best response directly, it is possible to define continuous dynamics for the game whereby each agent seeks to locally improve in the direction of steepest ascent of its value functions. Additionally, we can define an aggressiveness factor for each agent κj, which sets the rate at which an agent reacts to its value function gradient. The continuous dynamic in terms of bids is defined as TeX Source \eqalignno{\mathdot{w}_{j}(t)=&\,f_{j}\left(w(t)\right)=\kappa_{j}\left({\partial\over\partial w_{j}}v_{j}\left(w_{j}(t);\sum w_{-j}(t)\right)\right) \cr=&\,\kappa_{j}\left({c_{j}\sum w_{-j}(t)\over\left(\sum w_{j}(t)\right)^{2}}-1\right) &\hbox{(18)}}which is well defined for all ∑ wj > 0 and is defined as ∞ for ∑ wj = 0. Recall also that wj≥ 0, which introduces a discontinuity in the definition. The dynamics for the complete system are summarized as , and example trajectories are presented in Fig. 4 for the two-player game for varying levels of aggressiveness.

Fig. 4. Trajectories of discrete sequential and continuous dynamics for utility slope ratio c2/c1 = 6. Continuous trajectories are presented for aggressiveness ratios varying from κ21 = {1/40,1/3,1,3,40}. Best response curves are light solid lines, discrete trajectories are dashed lines, continuous trajectories are dark solid lines, the initial bids are denoted by a square, and the equilibrium is denoted by a diamond.

#### 1) Local Stability

The linearized continuous dynamics are . At w∗ = (c12 c2/(c1+c2)2, c1 c22/ (c1+c2)2), the eigenvalues of the linearized system can be shown to be negative for all values of κ12,c1,c2 > 0 but are omitted for space considerations.

#### 2) Convergence of Continuous Dynamic

To demonstrate convergence of the continuous dynamic, a series of definitions and theoretical results are required. This section relies on an established body of work in nonlinear analysis [21].

First, a trajectory is defined as γ(t,w0), where γ: +× 22, t+, and w02. An invariant set is one for which no trajectory that starts in S at t0 = 0 exits the set S. A point w is an ω-limit point of a trajectory γ if there exists a sequence of times tn such that tn→∞ as n→∞ for which limn→∞γ(tn,w0) = w and can be interpreted as any point that is visited infinitely often. An ω-limit set ω(w0) of point w0 is the set of all ω-limit points of the trajectory γ(t,w0). Closed orbits are defined as periodic trajectories that do not contain any equilibria. Finally, heteroclinic (respectively, homoclinic) orbits are defined as the union of multiple equilibria (respectively, a single equilibrium) and trajectories connecting them.

To complete this brief overview of the necessary concepts, two well-known results are restated. The first, known as Andronov's theorem, further characterizes ω-limit points.

#### Theorem 5 (Andronov's Theorem [22])

Every ω-limit point must be an equilibrium, an element of a closed orbit, or an element of a heteroclinic or homoclinic orbit.

Next, Bendixson's theorem can be used to determine the lack of existence of closed orbits inside a simply connected region.

#### Theorem 6 (Bendixson's Theorem [21])

Suppose is a simply connected region such that the divergence div(f) = (∂ f1/∂ w1)+(∂ f2/∂ w2) is not identically zero in D and does not change sign anywhere in D. Then D contains no closed orbits of .

It is possible to demonstrate, in three steps, convergence of the continuous dynamic defined in Section II-A over a set of initial conditions. The first step is to demonstrate that the set of feasible initial conditions defines an invariant set for any values of κ12 and for values of c2/c1 < 25/4. The second step involves showing that closed orbits do not exist inside the region of interest, which is done using Bendixson's theorem for planar systems. Finally, it is possible to demonstrate that the equilibrium is the unique element of the ω-limit point for all points inside the invariant set. By Andronov's theorem, it is known that all ω-limit points must be equilibria, closed orbits, or heteroclinic/homoclinic orbits. Closed orbits can be eliminated by use of Bendixson's theorem, heteroclinic orbits by the fact that only one equilibrium exists inside the invariant region, and homoclinic orbits by the local stability of the equilibrium. This guarantees that all trajectories must eventually reach the equilibrium. Since the equilibrium is locally stable, a trajectory cannot leave the equilibrium once it has been reached, and convergence is ensured.

An invariant set for the continuous dynamics can be defined by the set of feasible initial conditions, which is all points where both agents bid no more than the maximum of their best response functions (w1,w2)∊[0,c1/4]× [0,c2/4]. For cases where c2/c1 > 4, the region can be further restricted to ensure that neither agent is priced out of the market on the initial bid, namely, that w2∊[0,c1]. The region under consideration can be defined by four separate curves, as depicted in Fig. 5, labeled Ω ,and defined in Table 1.

Fig. 5. Invariant region for utility slopes c1 = 1, c2 = 2.
Table 1 Invariant Set Definition

#### Lemma 1

The region Ω defined in Table 1 is an invariant set for the system defined in (18).

Please refer to the Appendix for details of the proof.

It is now possible to state the main convergence result for continuous dynamics with two agents.

#### Theorem 7 (Convergence to Nash Equilibrium)

Given utility slope ratios of c2/c1∊[4/25,25/4], dynamics defined by (18), and any initial bid pair w0∊Ω, the system converges to the equilibrium TeX Source $$w^{\ast}=\left({c_{1}^{2}c_{2}\over(c_{1}+c_{2})^{2}},{c_{1}c_{2}^{2}\over(c_{1}+c_{2})^{2}}\right).$$Please refer to the Appendix for details of the proof.

Although this result is limited to utility slope ratios c2/c1 < 25/4, it is our conjecture that convergence holds beyond this threshold as well. The difficulty in extending the result lies in the discontinuity in the definition of the continuous dynamics when a zero bid is placed by either agent in reaction to a bid placed by one agent above the maximum at which the other agent will participate, which eliminates the ability to apply either Bendixson's or Andronov's theorem.

The analysis of convergence properties in this section reveals that the continuous update dynamics are stable for a much larger range of agents than iterative best response bidding. This result is in contrast with the desire of mechanism designers to allow the more natural mechanism definition where agents bid iteratively and react to current market prices. The lesson for the air traffic control application is therefore that if issues arise with limit cycles or zero bids in the iterative setting, convergence to the unique Nash equilibrium can be improved by approximating continuous ascent dynamics. In practice, this can be done by enforcing a maximum bid adjustment step size.

The methods employed for the convergence analysis rely heavily on the fact that only two agents were considered. In extending these results to more than two agents that exist for air traffic control, a novel approach such as a Lyapunov stability argument would most likely be needed. Nonetheless, the insight provided by the two agent scenario should be applicable to problems with more agents.

SECTION IV

## APPLICATION TO AIR TRAFFIC FLOW CONTROL

The lump-sum market mechanism described above can be seen to satisfy three crucial properties required for the inclusion of airline preference information in the air traffic flow control problem. First, an equilibrium solution is known to exist even when airlines anticipate the effect of their bids on prices. Secondly, this equilibrium is known to exhibit a worst case loss in efficiency of 25%, ensuring that strategic airline behavior will not result in arbitrarily poor system performance. Thirdly, the convergence results from the previous sections reveal that it is possible to achieve this equilibrium through a distributed bidding process. Although it is true that other properties for this mechanism can also be considered essential, these remain in the realm of future work.

This mechanism is applied to an air traffic network flow model first defined in [7]. The model aggregates flow separately for each airline between origin and destination pairs and seeks to avoid excessive densities of aircraft at any one resource (jetway or airport runway) at any specific point in the planning horizon, based on resource capacity estimates derived from weather forecasts. The flow control approach does not develop trajectories for individual aircraft but rather determines appropriate traffic levels for individual links to meet regional goals, such as rerouting traffic flows around congestion or weather. This approach is reasonable in light of the hierarchical structure of the NAS, where more localized tasks of aircraft separation assurance are handled by TRACON and sector controllers, while regional concerns are in the realm of traffic-flow management at the center level. Also, since aircraft operate most efficiently at their desired cruising speed, and since the flow abstraction necessarily reduces the model fidelity, the flows are assumed to operate at fixed velocities, which greatly reduces the computational complexity of the flow allocation problem.

It is important to note that other models exist for the NAS resource allocation problem [6], [23], which enable variable velocity flow modeling, or tracking of individual aircraft. This specific flow model was selected, however, for its convex and scalable formulation, which makes it ideally suited for use with the lump-sum market mechanism and for application to large portions of the NAS. This section proceeds by first presenting an overview of the preferred flow modeling approach. The lump-sum mechanism is then adapted to the network flow allocation setting, following its definition in [17], which is followed by a discussion of the computational complexity of the optimization required of the airlines. The mechanism is then implemented on a multiairport scenario in Section V.

### A. Network Definition

Consider an air traffic network defined by fixed sets of nodes n∊{1,⋖,N} = N and links i∊{1,…,I} = I. The node incidence matrix AN∊{0,1}N×N captures the connectivity of the network, with AabN = 1 if node a is connected to node b, and zero otherwise. Similarly, a link connectivity matrix AI∊{0,1}I× I is defined with AabI = 1 if link a flows into link b, and Aa,bI = 0 otherwise.

Fig. 6. A simple air traffic network with five links is used to illustrate the various components of the definition, where allowable flows are indicated by dashed and dotted lines. The source set is S = {1,2,3} and the sink set is Z = {4,5}. For this network, there are no merge sets, as all flow is split between multiple links. The divide sets D1 = D2 = D3 = {4,5} and inflow sets F4 = F5 = {1,2,3} capture the properties that flow from links 1–3 can be split into links 4 and 5.

### B. Discrete Path Flow Model

Instead of allowing complete authority over flow velocity and route selection, the path flow model restricts the flow to predefined paths with fixed velocity profiles. The result is that continuity is conserved by the path definition and that capacity constraints need only be ensured upon entry to a link. The formulation is further simplified by discretizing time with time step δ t, as t∊{0,δ t, …,T} = T, where T+ represents the finite time horizon of interest and NT = | T|. This allows for the definition of a finite set of available link/time pairs referred to as resources r = (i,t)∊ R = I× T, with R = | R| the total number of resources. Each resource has associated with it an en route capacity limit, summarized by an en route resource capacity limit vector Ce+R. Once again, the assumption that these limits can be determined safely and in advance is implicit in the definition of the capacity of resources.

In the Discrete Path Flow model, airlines plan flights for specific origin–destination (OD) pairs (no,nd)∊ . Each flight must follow a route over the network, which can be started at any point in the planning horizon (i.e., delayed arbitrarily). A route/time pair is referred to as a path p, which can be represented in terms of the resources the flights will consume, and implicitly defines the speed at which the aircraft travel over the network p = (r1,…,rNp), where Np is the length of the path. Note that each path is defined for a specific airline,2 so that pPj, , and P = |; P|. Each resource is assigned to the path at the time of entry onto a link, and therefore encapsulates the dynamics of the network flow in a matrix representation. The path-resource matrix Ae∊{0,1}R× P is defined as TeX Source $$A^{e}(r,p)=\cases{1,&path p consumes resource r\cr 0,&otherwise.}\eqno{\hbox{(19)}}$$Traffic flow along each path can now be allocated such that capacity limitations are observed. Define y+P as the flow allocation along all paths; then en route capacity limit constraints are Ae yCe.

Similarly, airport departure and arrival flow can be restricted to satisfy runway capacity limitations. Since multiple routes can be defined for each OD pair, airport arrival and departure flow can be determined by summing over the corresponding path flow allocations. Define the matrices Ψod,∊{0,1}V× P to map any path flow allocation y to origin–destination pair inflows yo+V, and outflows yd+V, where V = OD × T is the set of OD/time pairs, of cardinality V. Then TeX Source $$\Psi^{o}(v,p)=\,\cases{1,&path p starts at OD/time pair v\cr 0,&otherwise}\eqno{\hbox{(20)}}$$yo = Ψo y and yd is defined similarly. The airport departure and arrival flow can be defined by summing over all OD pairs that arrive at or depart from a specific airport. Similar to (19), let be the origin and destination matrices, respectively. Then airport capacities are constrained by Ao yoCo and Ad ydCd, where Co|||S||× NT and Cd|||Z||× NT are the limits on safe airport operation.

### C. Discrete Airline Model

In order to react strategically to unpredictable phenomena such as weather, it is necessary to define an airline cost function over possible outcomes. For airlines, aircraft arrival delays relative to the predefined flight schedule are an excellent indicator of the costs to be incurred by air traffic control decisions, and are in fact monitored by the FAA to assess air traffic control performance [1]. Therefore, the airline costs are modeled by minimizing the quadratic deviation from the cumulative scheduled arrival flow, which penalizes not only deviations from the scheduled flow but also how long those deviations persist. This cost structure does not exactly penalize individual flight delays, however, as this information is lost in the flow aggregation process. The quadratic nature of the cost function does penalize large deviations from the flight schedule more heavily than small ones, which mimics the fact that larger disruptions tend to impact subsequent scheduled flights more severely, driving a domino effect of cost escalation as delays persist. Although deviations from the planned routes, departure delays, and en route velocity changes can impact airline costs, it is reasonable to assume that arrival delays are the most prominent concern. However, concave, nondecreasing cost functions can easily be designed that also penalize additional quantities such as cumulative deviation from the scheduled departure flow or deviation from scheduled path flows. Future work will aim to include the case of airline flight cancellations as well, which essentially define an outside option as resource prices increase.

To simplify notation, let Yo+V and Yd+V be the cumulative origin inflow and destination outflow for all OD/time pairs, respectively. Cumulative departure inflow is defined by Yo = Γ yo, where Γ∊{0,1}V× V is a block diagonal matrix with lower triangular blocks of dimension NT, and cumulative arrival inflow is defined similarly. Let yso be the scheduled departure flow and ysd be referred to as the scheduled arrival flow, with Yso and Ysd defined equivalently to Yo and Yd. A representative airline cost function can be defined as TeX Source $$J_{j}(y_{j})=\left\Vert\Omega_{j}\max\left({\bf 0},Y_{j}^{sd}-Y_{j}^{d}\right)\right\Vert_{2}^{2}\eqno{\hbox{(21)}}$$where Ωj+V× V is a diagonal weighting matrix that captures each airline's preferences over OD pairs at each time epoch. Note that the zero vector of dimension V is represented by 0 and the max function is performed elementwise; that is, max (0,·) operates on a vector such that positive entries are unchanged and all other entries are set to zero. Equation (21) defines an example of an airline cost function that is convex and nonincreasing, since it is a positive linear combination of convex, nonincreasing functions in the path flow variables yj.

### D. Centralized Optimization

It is now possible to define a centralized optimization program with complete information of airline preferences. Let A and C be defined as TeX Source $$A=\left[\matrix{A^{e}\cr A^{o}\cr A^{d}}\right],\quad C=\left[\matrix{C^{e}\cr C^{o}\cr C^{d}}\right].\eqno{\hbox{(22)}}$$Note that the matrix A can be divided into NJ submatrices A = [A1|…| ANJ], which map airline path flow allocations yj to the resources consumed. The air traffic path flow allocation problem can now be seen to take a similar form to the central allocation problem defined in Problem P2.1 TeX Source \matrix{\noalign{\hrule\vskip 6.5pt}\smash{\vrule depth 40pt height 42pt}&\eqalign{&\hbox{Central Path Flow Allocation}\cr\noalign{\vskip 3pt} &{\hskip 20pt}\buildrel{\hbox{maximize}}\over{\scriptstyle{{y\geq 0}}}\quad{\hskip 1pt}\sum\limits_{j\in{\cal J}}J_{j}\cr\noalign{\vskip 4pt}&{\hskip 20pt}\hbox{subject to}\quad Ay\leq C}\qquad{\hbox{(P4.1)}}&\smash{\vrule depth 40pt height 42pt}\cr\noalign{\vskip 9pt\hrule}}whose solution yc, represents the ideal solution to the multiairline resource allocation problem if airline preferences were public information. The convex formulation ensures that for problems of reasonable size, the optimization can be performed conveniently using standard convex optimization tools such as CVX and SeDuMi in the Matlab environment [24], [25], [26].

### E. Lump-Sum Market Mechanism for Air Traffic Flow Control

The definition of the lump-sum market for this problem formulation requires additional information of the bidders, in order to avoid situations where a Nash equilibrium might not exist if demand for a resource is insufficient to keep prices away from zero [17]. Bidders are also required to submit a link flow request , which is the amount of flow they would like to have allocated when the price for a resource is zero. Let be defined as the airline strategy, with z = (z1,…,zJ) the complete strategy vector. The flow allocation process is then augmented for each resource as follows: TeX Source $$x_{ij}(z)=\cases{{w_{ij}\over\sum\limits_{j\in{\cal J}}w_{ij}}C_{i}&\sum\limits_{j\in{\cal J}}w_{ij}>0\cr\mathhat{x}_{ij}&\sum\limits_{j\in{\cal J}}w_{ij}=0,\sum\limits_{j\in{\cal J}}\mathhat{x}_{ij}\leq C_{i}\cr 0&\sum\limits_{j\in{\cal J}}w_{ij}=0,\sum\limits_{j\in{\cal J}}\mathhat{x}_{ij}>C_{i}}\eqno{\hbox{(23)}}$$where xij(z) is the allocation of flow on link i assigned to airline j resulting from the airline strategy z. Note that if the total requests for flow on a link exceed the link capacity but the sum over all bids on that link is zero, no airline receives any flow allocation as the link has become contested.

The local airline cost functions are defined as TeX Source $$J_{j}^{\rm PA}(z_{j};z_{-j})=J_{j}\left(y_{j}\left(x_{j}(z)\right)\right)+\sum_{i\in{\cal I}}w_{ij}.\eqno{\hbox{(24)}}$$Note that the airlines not only determine the lump sum bids, wj, to place for each resource, but also identify the path flows, yj, which maximize the value of the flow allocations over each of the links, xj, which are determined by the lump sum bids. In effect, the requirement to identify path flows in addition to resource bids represents an embedded network flow maximization problem in the argument of the airline cost.

The definition of a Nash equilibrium of (5) is augmented to include the flow requests TeX Source $$J_{j}^{\rm PA}\left(z_{j}^{\rm NE};z_{-j}^{\rm NE}\right)\leq J_{j}^{\rm PA}\left(z_{j};z_{-j}^{\rm NE}\right),\quad\forall z_{j}\in\BBR_{+}.\eqno{\hbox{(25)}}$$

The existence of a Nash equilibrium and its efficiency loss can now be established for the multilink network problem. Because the strategy space of the airlines has been expanded to include link flow requests, the requirements on the costs can be relaxed to no longer require decreasing, differentiable costs, although the uniqueness of an equilibrium can no longer be guaranteed. In establishing these results for airlines with costs instead of utilities, the following assumption is required.

#### Assumption 1

The airline cost functions Jj are assumed to be convex, nonincreasing, and continuous. Furthermore, the cost of a null allocation for each resource is assumed to be finite, and is referred as the worst case cost, dj = Jj(0).

As a result of the worst case cost function, the airline cost functions can be posed as utility functions Uj(xj(z)) = djJj(xj(z)), which are nonnegative, concave, continuous, and nondecreasing for all flow allocations, and evaluate to Uj(0) = 0 for a null allocation. This assumption allows for a modest generalization of the results of [17] to problems in which airlines measure preference in terms of cost, and for which null allocations do not result in zero costs/utilities.

#### Theorem 8 (Existence [17])

Assume that for J > 1, each airline j has a cost function Jj that satisfies Assumption 1. Then there exists a Nash equilibrium zNE≥ 0 of the game defined by (J1PA,…,JJPA) if the allocation is defined by (23).

The efficiency loss bound for lump-sum markets also holds in the current context.

#### Theorem 9 (Efficiency Loss [17])

Let zS be the unique efficient solution to the central air traffic control allocation problem and zG be a Nash equilibrium of the game (J1PA,…,JJPA). Then TeX Source $$\sum_{j\in{\cal J}}d_{j}-J_{j}\left(x_{j}^{G}(z^{G})\right)\geq{3\over 4}\left(\sum_{j\in{\cal J}}d_{j}-J_{j}\left(x_{j}^{S}(z^{S})\right)\right).\eqno{\hbox{(26)}}$$Furthermore, for any set of airlines J, there exist linear utilities for which this bound is tight.

The local airline optimizations can now be defined for both price-taking and price-anticipating airlines. For airlines that act as price-takers, the airline subproblems are defined as TeX Source {\hskip-2pt}\matrix{\noalign{\hrule \vskip 8.5pt}{\hskip -0.01pt}\smash{\vrule depth 37pt height 41.5pt}&{\hskip -5pt}\eqalign{&\hbox{Price-Taking Airline Subproblem}\cr\noalign{\vskip 3pt} &{\hskip 10pt}\hbox{minimize}\quad{\hskip 1pt}J_{j}(y_{j})+\sum\limits_{i\in{\cal I}}w_{ij}\cr\noalign{\vskip 4pt}&{\hskip 10pt}\hbox{subject to}\quad \lambda_{i}A_{i,j}y_{j}\leq w_{ij},\quad\forall i\in \cal I.}\qquad{\hbox{(P4.2)}}&{\hskip -11.5pt}{\hskip 11.1pt}\smash{\vrule depth 37pt height 41.5pt}\cr\noalign{\vskip 9pt\hrule}}The notation Ai,j denotes the ith row of the jth sub-matrix, Aj, of the matrix A defined in (22). Ai,j is a row vector of the same length as yj, the path flow column vector.

For airlines that act as price-anticipators, a simplifying assumption is made that there exists a minimium bid, ∊w > 0, for every resource, ensuring that only the first pricing rule in (23) is applicable. The airline subproblems are defined as TeX Source \matrix{\noalign{\hrule\vskip 6.5pt}\smash{\vrule depth 52pt height 54pt}&\eqalign{&\hbox{Price-Anticipating Airline Subproblem}\cr\noalign{\vskip 3pt} &{\hskip 10pt}\hbox{minimize}\quad{\hskip 1pt}J_{j}(y_{j})+\sum\limits_{i\in{\cal I}}w_{ij}\cr\noalign{\vskip 4pt}&{\hskip 10pt}\hbox{subject to}\quad A_{i,j}y_{j}\leq C_{i}\left({w_{ij}\over \sum\limits_{j\in \cal J} w_{ij}}\right),\quad\forall i\in \cal I.}\quad{\hskip-14pt}{\hbox{(P4.3)}}&\smash{\vrule depth 52pt height 54pt}\cr\noalign{\vskip 9pt\hrule}}In both cases, the airline subproblem is a convex optimization program and can therefore be solved efficiently with standard optimization tools. The airline cost functions are convex, and for Problem P4.2, the constraints are linear. The flow constraints in Problem P4.3 are also convex, which can be seen by rewriting them as TeX Source $$A_{i,j}y_{ij} \leq C_{i}\left(1-{\sum_{k\neq j}w_{ik}\over {w_{ij}+\sum_{k\neq j}w_{ik}}}\right)\eqno{\hbox{(27)}}$$and noting that for constant a ∊ (0, ∞), w ∊ (0, ∞), the function f(w) = a/(w+a) is convex in w.

The lump-sum market mechanism is adapted for the air traffic control problem in Algorithm 1, where iterations are indexed by k.

Algorithm 1 Discrete dynamic algorithm for lump-sum markets
1: Initial bids, zj0, submitted .
  2: k = 0
  3: repeat
  4: λk = ∑ wjk/C, central price update.
  5: Allocations xjk(z)) are set as defined in (23).
  6: Agents update their responses zjk+1 by solving Problem P4.2 or P4.3.
  7: until ∑‖ zjk+1−zjk‖ < ∊z

### F. Problem Size

The size of the air traffic flow control problem defined above depends primarily on the length of the planning horizon and the subset of the U.S. NAS to be included. The U.S. NAS contains 60 major airports3 for which flow scheduling might be considered, which results in as many as 3540 OD pairs. The total number of link resources can be approximated to be 2500 by assuming approximately seven links across each of 400 sectors that cover the United States. For a path flow model with five alternate routes per OD pair and 5 min time intervals, the flow allocation problem could reach sizes on the order of one million flow variables and one-half million resource constraints. At this size, exact second-order techniques are currently impractical, and convex program solution times in the tens of minutes for each local optimization are likely on standard computing hardware. More tractable problems can be generated by including only those resources for which congestion or weather have resulted in a desired schedule that currently strains capacity limits. Similarly, the problem size is significantly improved by limiting the planning time horizon to three hours, which is in line with the time horizon for which sufficiently accurate weather predictions are available. The simulation results presented in Section V are representative of problem sizes for which real-time market-based allocation converges in a realistic time-frame for mitigating schedule disruptions due to weather.

SECTION V

## SIMULATION: AIR TRAFFIC FLOW CONTROL

### A. Scenario

A scenario based on the northeastern United States is used to investigate the relative performance of price-taking and price-anticipating airlines in the lump-sum market mechanism. For this scenario, flow departing from seven airports within approximately three hours of and destined for New York is modeled, as depicted in Fig. 7(a). Once again, the simulation spans a 3-h time horizon and uses a 5-min time step. The nodes of the network are based on actual airport and airway intersection locations available from the FAA,4 links were generated artificially for convenience only, and a primary shortest path and secondary alternate path were generated over the network for each OD pair.

Fig. 7. Aggregate flow of traffic to New York airports for all airlines. Each figure has airports marked with black circles, and all airway intersection nodes depicted with diamonds. (a) presents the network map. (b)–(d) depict flow volume at times t = 50, 100, 150 min and use link width to represent flow volume. The weather disruption polygon is shown in red (dark gray), as are all links and nodes affected by weather.

Four airlines are considered for this scenario, as summarized in Table 2. They are defined to represent the various types of airlines in existence, including low-cost and mainline carriers, as well as large- and small-volume airlines. The schedule definitions for mainline carriers tend to be peaked, meaning that many flights are scheduled to arrive at their hub airports at the same time, putting a strain on the airport and increasing their cost of operations. In contrast, low-cost airlines try to maintain flat schedules that utilize gates and surface crews evenly, thereby reducing their costs.

Table 2 Scenario: Airline Schedules and Flow Preferences

En route capacity limitations are uniformly set to 90% of the peak scheduled flow along the busiest link, and inclement weather is included as a polygon with fixed size and a known velocity that reasonably approximates standard weather pattern movements. Capacity is assumed to be restricted by 50% for all flows with any subsection of the link inside the weather polygon at each time step.

### B. Results

Fig. 7(b)–(d) shows the resulting aggregate flow over the network for all airlines at a sequence of times. The weather polygon and affected nodes are also shown, and link width is again used to represent flow volume along a link. The weather disruption polygon and affected links and nodes are represented in red (dark gray).

Fig. 7(b) shows the relatively low-demand starting period of the simulation, where most flows are taking the shortest path routes to the destination and none of the flow is close to the link capacity limits. Once again, weather has a significant impact on flow to the destination; in particular, Fig. 7(d) shows a surge of recovery flow once the weather has passed.

Fig. 8 presents the arrival traffic and pricing throughout the simulation for both price-taking and price-anticipating airlines, as well as the resulting flow and constraint satisfaction. From this figure, one can see the flow at the destination airport being reduced during the weather disturbance, then recovering once the disturbance has passed. Note that the recovery does not occur immediately, as not only the destination airport but also the links leading to the destination were affected by the weather disruption, so recovery was stalled until flows along the final legs of the flight paths could again reach the airport.

Fig. 8. Arrival flow at the destination airport. (Top) Arrival flow quantities: scheduled arrival traffic (light green line), airport capacity (dark red line), price-taking solution (dark blue o), and price-anticipating solution (light magenta ⋄). (Bottom) Airport pricing for price-taking solution (dark blue o) and price-anticipating solution (light magenta ⋄).

The main distinction in the solutions of the price-taking airlines and the price-anticipating airlines is that the prices that result in the price-anticipating solution are lower. This effect is caused by the airlines that command a significant portion of the available resources, which bid lower than they otherwise would in order to keep prices down. In the extreme with two price-anticipating airlines and a single resource, if one airline bids zero, the other is incentivized to bid as close to zero as possible, as it will still receive all the available resource regardless of the magnitude of its bid. By contrast, the price-taking airline would bid such that the price equals the negative of its marginal cost when it receives all the available resource, which is some finite nonnegative value for strictly increasing, convex cost functions.

The deviation from the desired cumulative schedule is plotted for each airline in Fig. 9, for both the price-taking and price-anticipating airlines. The high-cost high-volume airline (Airline 1) has increased its deviations in the price-anticipating solution, whereas the other airlines (Airlines 2–4) have decreased their deviation. Essentially, the high-cost high-volume carrier commands enough of the available resources to depress market prices to its advantage, and so incurs some additional delay as a result.

Fig. 9. Deviation from desired cumulative arrival flow: price-taking solution (solid blue line) versus price-anticipating solution (dashed magenta line) for each airline. The high-cost high-volume airline (Airline 1) incurs more delay as a price-anticipator, and other airlines (Airlines 2–4) incur less delay.

A comparison of price-taking and price-anticipating airline costs is presented in Table 3, where columns represent the actual delay cost incurred, the resulting payments made by the airlines, and the total market cost that results for each airline. Three interesting elements of the results are visible in this table. First, the overall efficiency of the solution is better with price-taking airlines versus price-anticipating airlines, as visible from the total delay cost for each solution. This follows from the fact that the lump-sum market with price-taking airlines converges to the efficient solution, whereas the Nash equilibrium for price-anticipating airlines need not be efficient. Secondly, since the resource prices are lower with price-anticipating airlines, so too are the payments they make (for all but Airline 4). Lastly, all airlines prefer their market totals in the price-anticipating scenario over those in the price-taking scenario.

Table 3 Cost Comparison and Resource Payments for All Airlines if Acting as Price-Takers or Price-Anticipators

Finally, the market mechanism convergence performance can be observed in Fig. 10. The slow rate of convergence of the market mechanism with price-taking airlines is a direct result of the pricing update mechanism, which must maintain a small step-size to avoid excessive oscillation of market prices. Subgradient techniques can be used for the price update, but the algorithm can only be guaranteed to converge in an unbounded number of steps [27]. In contrast, with price-anticipating airlines, there is a significant improvement in the number of iterations required for the market prices to converge, which is indicative of the anticipatory effect that airlines have by including the effect of their bids on market prices in their local decision-making. In practice, the local optimizations involve convex nonlinear constraints on the airline bids, and therefore take longer to perform than the equivalent price-taking optimization, so some of the benefit of the faster market convergence is lost. For a market mechanism that requires hundreds of iterations to converge, local airline optimizations cannot consume longer than 10 s if time constraints on the overall computation are to be met.

Fig. 10. Convergence of lump-sum market based flow allocation with the discrete path flow model for both price-taking (dotted blue) and price-anticipating (solid magenta) airlines. Error as a percentage of the final cost.
SECTION VI

## CONCLUSION

In summary, this paper proposes the use of a lump-sum market mechanism for the allocation of resources amongst price-anticipating airlines competing for air traffic network flow, in order to improve system efficiency when weather disruptions affect capacity. This method allows for the fact that airlines are in direct competition and will attempt to gain an advantage over their competitors.

This paper includes an investigation of convergence properties of the mechanism to strengthen the justification for its implementation. By restricting the class of problems to two agents with linear utilities, both discrete best response and continuous steepest descent dynamics can be classified into regions of guaranteed stability. Most interestingly, the continuous dynamic is stable over a broader range of agent types than the discrete dynamic, which implies that for systems where convergence of the discrete best response dynamic is too slow or nonexistent, a limited step-size approach may improve the situation.

The effect of price-anticipation by the airlines on the market outcome is demonstrated for a representative flow problem, and both moderate efficiency loss and with lower resource pricing result, and in fact all airlines prefer their outcome with this mechanism over the efficient market solution that results from price-taking strategies. Although no convergence guarantee exists for price-anticipating airlines on network flow problems, in practice, the convergence results were in fact significantly better when airlines predict the effect of their bids on prices.

### A. Future Work

Many fascinating aspects of the implementation of market mechanisms for disruption management in air traffic control remain. The notion that airspace resources have inherent value, and that allocation to the highest bidder is the most desirable method for managing excess demand, is certainly not universally accepted. Great care will need to be taken to address fears from players such as the general aviation community, but ultimately both safety and efficiency concerns will be advanced if busy routes and airports are preferentially reserved for larger vehicles with higher resource valuations. Similarly, safety concerns may result if convergence to feasible solutions cannot be guaranteed within the planning window, and further work is needed in defining market performance guarantees before their use in safety-critical applications can be considered. The CDM initiative provides a useful benchmark for how to appropriately implement a market mechanism with airline participation. The ration by schedule approach for GDPs, which assigns delays based on the existing flight schedule, results in an incentive for airlines to provide accurate flight information, thereby reducing the frequency of unused landing slots. Although the mechanisms are different, the lump-sum market approach proposed in this paper does not inherently give airlines resource rights implied by the existing flight schedule. It is important for future work to address this limitation of the lump-sum market theory to ensure various stakeholders have proper incentives and are satisfied with the equity of the approach to ensure participation in a coordinated flow control solution.

Specific to the market mechanism presented in this paper, it is important to note that the price-anticipating airline model does not capture every possible action that airlines might take to attempt to manipulate the allocation mechanism. Specifically, the repeated nature of the allocation problem allows many additional strategic behaviors to be executed by the airlines that are not captured in this model. Well-known negative results demonstrate that every allocation can be sustained as an equilibrium in repeated games, through strategies that punish deviation over multiple epochs. Even in the single market case, each set of bids submitted by the airlines is evaluated separately in this model, without any notion that multiple bidding rounds are required for convergence. This is a strict limitation on the strategies considered for the airlines, and may therefore not capture the true outcome if the mechanism were implemented.

From the air traffic flow modeling perspective, further issues also remain. In particular, the conversion of the equilibrium flow solution into modifications to the existing aircraft flight plans has not been addressed. The definition of the network flow problem is inherently continuous, which is a requirement of the mechanism under consideration, whereas the existence of individual flights imposes an integer constraint on modifications. The result is that the flow solution may be difficult to achieve in practice, despite the fact that airlines are required to make payments for their flow allocations. In each case, these issues represent interesting areas of future research, with broad implications not only for air traffic management but also for many other multiagent engineering applications.

## APPENDIX

First, the proof for Theorem 4 is presented.

#### Proof

First, there exists a region Λ0 of increasing bids, where both agents will always increase bids until Λ0 is exited. This region can be identified by inspecting the best response functions and noting that β2(w1) is increasing in w1 throughout Λ, while β1(w2) is increasing for w2∊(0,c1/4]. Therefore, any bid pair starting in the region (0,c1/4]×(0,c1/4] must result in a trajectory that exits Λ0 at some step k.

For the remainder of the feasible set (0,c1/4]×(c1/4,c2/4], the worst case slopes in terms of the magnitude of the eigenvalues of a linearized dynamic occur at w1 = ∊ > 0, where ∂β1/∂ w2→−1/2 as ∊→ 0 and w2 = β2(c1/4), where . The resulting eigenvalues of this worst case linearization are uniformly bounded inside the unit circle for all c2/c1 < 9/4. Therefore, all trajectories that start in the feasible set Λ converge to the equilibrium w∗ for ratios of c2/c1 < 9/4 and c2/c1 > 1.

Next, the proof for Lemma 1 is presented.

#### Proof

Invariance is demonstrated by taking the dot product of the outward normal to the boundary and the dynamics, which is shown to be every nonpositive. Along g1 TeX Source \eqalignno{(\mathdot{w}_{1},\mathdot{w}_{2})\cdot(-1,0)=&-\mathdot{w}_{1}=-\kappa_{1}\left({c_{1}\over w_{2}}-1\right)\leq 0,\ \ \cr&\forall \ w_{2}\leq c_{1}.&{\hbox{(28)}}}The remaining curves can be evaluated similarly.

Finally, the proof for Theorem 7 is presented.

#### Proof

The region Ω defined in Fig. 5 is a simply connected region that is invariant to the flow defined in (18) for ratios of cj/ck∊[1,25/4]. Therefore, any trajectory starting from a point w0∊Ω cannot exit Ω and must spend infinite time in the region. By Andronov's theorem, every ω-limit point must be an equilibrium, an element of a closed orbit, or an element of a heteroclinic or homoclinic orbit. There exists only one equilibrium point in Ω, as shown in Section III-B, and it is stable; therefore heteroclinic and homoclinic orbits are not possible in Ω, and so all ω-limit points must be either elements of a closed orbit or the unique equilibrium. The existence of closed orbits can be ruled out using Bendixson's theorem since the divergence of the flow field is negative for all w∊Ω TeX Source \eqalignno{div(f)=&\,-2\left({\kappa_{1}c_{1}w_{2}\over(w_{1}+w_{2})^{3}}+{\kappa_{2}c_{2}w_{1}\over(w_{1}+w_{2})^{3}}\right)\cr<&\,\ 0,\quad\forall\, \{w_{1},w_{2}\}\in\Omega.&\hbox{(29)}}Therefore, the unique ω-limit point in Ω is the equilibrium, and so all trajectories starting in Ω must converge to this equilibrium. Since the equilibrium is locally stable, a trajectory cannot leave the equilibrium once it has been reached.

## Footnotes

Manuscript received nulldate; revised April 07, 2008. Current version published nulldate.

S. L. Waslander is with the Department of Aeronautics and Astronautics, Stanford University, Stanford, CA 94305 USA (e-mail: stevew@stanfordalumni.org).

K. Roy is with the Department of Electrical Engineering, Stanford University, Stanford, CA 94305 USA (e-mail: kroy1@stanford.edu).

R. Johari is with the Department of Management Science and Engineering and the Department of Electrical Engineering, Stanford University, Stanford, CA 94305 USA (e-mail: johari@stanford.edu).

C. J. Tomlin is with the Department of Electrical Engineering and Computer Sciences, University of California at Berkeley, Berkeley, CA 94720 USA and the Department of Aeronautics and Astronautics and Hybrid Systems Laboratory, Stanford University, Stanford, CA 94305 USA (e-mail: tomlin@eecs.berkeley.edu).

2. For the path flow model, the subscript j shall be used to denote the components of a set, vector, or matrix that corresponds to the jth airline.

3. A major airport is defined as one that supports > 0.25% of total passenger enplanements in the United States.

4.

## References

1. Bureau of Transportation Statistics

Department of Transportation, 2006, http://www.bts.gov/

2. FACET: Future ATM concepts evaluation tool

K. Bilimoria, B. Sridhar, G. Chatterji, K. S. Sheth, S. Grabbe

Air Traffic Contr. Quart., Vol. 9, issue (1), pp. 1–20, 2001-03, http://www.technology.arc.nasa.gov/SOY2006/SOY_FACET/

3. Slot trading opportunities in collaborative ground delay programs

T. V. Vossen, M. O. Ball

Transp. Sci., vol. 40, issue (1), p. 29–43, 2006-02

4. The air traffic flow management problem with enroute capacities

D. Bertsimas, S. S. Patterson

Oper. Res., vol. 46, issue (3), p. 406–422, 1998-05/06

5. A new approach for modeling, analysis and control of air traffic flow

P. Menon, G. Sweriduk, K. Bilimoria

Monterey, CA
Proc. AIAA Guidance, Navig. Contr. Conf. Exhib., 2002-08

6. Eulerian trilogy

D. Sun, S. D. Yang, I. Strub, A. M. Bayen

Keystone, CO
Proc. AIAA Guidance, Navig. Contr. Conf. Exhib., 2006-08

7. Market-based air traffic flow control with competing airlines

S. L. Waslander, R. L. Raffard, C. J. Tomlin

AIAA J. Guidance, Dyn. Contr., vol. 31, issue (1), p. 148–161, Jan.–2008-02

8. Auctions for the safe, efficient and equitable allocation of airspace system resources

M. O. Ball, G. L. Donohue, K. Hoffman

Combinatorial Auctions, P. Cramton, Y. Shoham, R. Steinberg, Cambridge, MA: MIT Press, 2006, ch. 22, pp. 507–538

9. Airport Slots: International Experiences and Options for Reform

A. Czerny, P. Forsyth, D. Gillen, H.-M. Neimeier

Aldershot, U.K.: Ashgate, 2008

10. Optimization and mediated bartering models for ground delay programs

T. Vossen, M. Ball

Naval Res. Logist., vol. 53, p. 75–90, 2006

11. An overview of pricing concepts for broadband IP networks

M. Falkner, M. Devtsikiotis, I. Lambadaris

IEEE Commun. Surv., 2nd quarter 2000

12. Architecture of power markets

R. Wilson

Econometrica, vol. 70, issue (4), p. 1299–1340, 2002-07

13. Microeconomic Theory

A. Mas-Colell, M. D. Whinston, J. R. Green

2nd ed., New York: Oxford Univ. Press, 1995

14. Strategic buyers in a sum-bid game for flat networks

B. Hajek, S. Yang

Proc. IMA Workshop 6: Contr. Pricing Commun. Power Netw., 2004-03

15. A scalable network resource allocation mechanism with bounded efficiency loss

R. Johari, J. N. Tsitsiklis

IEEE J. Sel. Areas Commun., vol. 24, p. 992–999, 2006-05

16. Rate control for communication networks: Shadow prices, proportional fairness and stability

F. Kelly, A. Maulloo, D. Tan

J. Oper. Res. Soc., vol. 49, issue (3), p. 237–252, 1998-03

17. Efficiency loss in a network resource allocation game

R. Johari, J. N. Tsitsiklis

Math. Oper. Res., vol. 29, issue (3), p. 407–435, 2004-08

18. Do greedy autonomous systems make for a sensible internet?

B. Hajek, G. Gopalakrishnan

Stanford, CA
Proc. Conf. Stochast. Netw., 2002

19. Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria

J. S. Shamma, G. Arslan

IEEE Trans. Autom. Control, vol. 50, p. 312–327, 2005-03

20. The Theory of Learning in Games

D. Fudenberg, D. K. Levine

Cambridge, MA
MIT Press, 1998

21. Nonlinear Systems: Analysis, Stability, and Control

S. Sastry

New York: Springer-Verlag, 1999

22. Nonlinear Oscillations, Dynamical Systems and Bifurcations of Vector Fields

J. Guckenheimer, P. Holmes

Berlin, Germany: Springer-Verlag, 1983

23. The traffic flow management rerouting problem in air traffic control: A dynamic network flow approach

D. Bertsimas, S. S. Patterson

Transp. Sci., vol. 34, issue (3), p. 239–255, 2000-08

24. CVX ver. 0.85

M. Grant, S. Boyd, Y. Ye

25. SeDuMi: Self dual minimization ver. 1.1

J. F. Sturm, O. Romanko, I. Polik

26. Matlab ver. 7.1.0

Mathworks, 2006, http://www.mathworks.com/products/matlab/

27. Minimization Methods for Non-Differentiable Functions

N. Z. Shor

Springer Computational Mathematics 3, K. C. Kiwiel, A. Ruszczinski, New York: Springer-Verlag, 1985

## Cited By

No Citations Available

## Keywords

### IEEE Keywords

No Keywords Available

### INSPEC: Controlled Indexing

air traffic control, game theory, optimisation, travel industry

### More Keywords

No Keywords Available

No Corrections

## Media

No Content Available
This paper appears in:
Proceedings of the IEEE
Issue Date:
December 2008
On page(s):
2113 - 2130
ISBN:
0018-9219
Print ISBN:
N/A
INSPEC Accession Number:
10401576
Digital Object Identifier:
10.1109/JPROC.2008.2006197
Date of Current Version:
16 Jan, 2009