Air traffic management for the U.S. National Airspace System (NAS) is one of the essential roles undertaken by the Federal Aviation Administration (FAA) in maintaining safe commercial flight operations. The air traffic control system is responsible for ensuring that resource limits such as airport takeoff and landing rates are not exceeded, as well as ensuring that minimum en route aircraft separation is maintained for up to 45 000 flights a day [1]. Typical operations, as depicted in Fig. 1, can require simultaneous management of upwards of 6000 aircraft. This complex task is aggravated by adverse weather conditions caused by inclement weather, which are difficult to predict beyond three hours in advance, and complicated by the many stakeholders for whom disruptions result in significant financial burdens. Delays incurred by any individual flight can result in repercussions throughout the network, as aircraft, crews, and passengers may all be scheduled to continue on subsequent flights. The result is a highly connected, dynamic, and unpredictable resource allocation problem with significant financial ramifications for the airlines.

The FAA has three main avenues for reconfiguring scheduled operations in the face of airspace disruptions: ground delay programs (GDPs), miles-in-trail restrictions, and flight reroutes. GDPs can be used to delay network traffic bound for specific constrained resources, such as destination airports and en route airspace. Miles-in-trail restrictions specify en route spacing on crowded airways to smooth flow into intersections and congested areas. Flight reroutes ensure aircraft are redirected around weather disruptions and are managed through a national *playbook* of fixed procedures developed through years of experience. Currently, these methods address multiple stresses to the system individually. NAS-wide coordination of these actions is managed manually at the FAA Air Traffic Control System Command Center (ATCSCC), with verbal input from members of the Collaborative Decision Making (CDM) initiative,^{1} which currently enables a common situational awareness as well as airline participation in GDPs. The CDM initiative is especially important as a model program because it is the first for which incentives were designed to enhance airline participation [3], since rationing resources by schedule eliminates the penalty formerly faced by airlines when providing accurate information about flight changes.

As demand for resources increases, however, the need for the ATCSCC to provide efficient coordinated national flow control decisions will only increase, and the research community has therefore proposed a variety of air traffic modeling and control techniques that enable NAS-level planning in real time [4], [5], [6], [7]. By aggregating individual flight plans into flows over a fixed network and defining capacity limits in both en route and terminal areas in terms of flow volumes, significant reductions in computational complexity are achieved over more detailed models that manage all flights individually. Though the ability to synthesize trajectories for each aircraft is lost, appropriate traffic levels for sectors are computed to meet regional demands due to congestion and weather. This approach is in line with the hierarchical structure of the NAS, in which the discrete problem of aircraft separation is handled by sector controllers, while air traffic management at the center level often deals primarily with regional flow concerns. Another issue arises due to the fact that the costs incurred when flights must be delayed are primarily borne by the airlines, which are in the best position to assess the relative value of favoring one set of flights over another. Recent advances have focused on long-term allocation of airport slot resources [8], [9], or specifically on the GDP process [10], but do not consider disruptions to daily operations due to weather. To achieve efficient utilization of airspace resources, up-to-date private airline cost information must be included in the traffic-flow decision-making process, a need that is complicated by the competitive nature of airline operations.

In many engineering applications, such as Internet routing and electricity generation, market mechanisms have been proposed as a way to incorporate preference information into resource allocation decisions with competing agents [11], [12]. Market mechanisms assign an explicit price to each resource, and agents balance the cost of purchasing resources with the value of those resources to them [13]. At each step in the process, the agents seek to minimize their costs by purchasing resources at market prices, after which the central coordinator updates resource prices in order to penalize excess demand. In previous work [7], a network flow model and market mechanism were presented for the air traffic control system, allocating airspace resources on a three-hour time horizon. The resources include take off, en route and landing traffic flows, and are allocated based on airline bids and centrally defined prices. Such a system could be implemented as a natural extension to the existing Collaborative Decision Making communications framework by requiring airlines to submit bids for traffic flow in areas affected by capacity restrictions due to weather.

Analysis of market mechanisms often relies on the assumption that agents treat prices as exogenous. For large numbers of agents with limited market share, such an assumption is reasonable. Unfortunately, applications with a small number of agents that command significant fractions of the available resources, such as the air traffic control problem, are susceptible to price manipulation by the agents. It becomes implausible to accept the assumption that agents would act without knowledge of how their actions affect market prices. Instead, agents must be thought of as *price-anticipating,* which is to say, capable of predicting the effect of changes in their own requests on the market. This can lead to arbitrary efficiency loss for certain market mechanisms [14], [15] and requires the definition of an alternate mechanism for which such a negative result can be avoided.

One such mechanism exists in the form of the lump-sum market, which was recently studied in the context of Internet congestion control [16], [17]. Instead of having agents request resource allocations directly, the lump-sum market, as its name suggests, requires agents to specify lump-sum payments they are willing to make for each resource, resulting in an elastic demand that can always be satisfied by adjusting prices relative to the cumulative payment per resource. Existence and uniqueness results for a Nash equilibrium have been established for price-anticipating agents in the single resource case [18], and existence of Nash equilibria in the case of multiple network resources [17]. What makes this market mechanism appealing is that a proportional bound on efficiency loss relative to the central solution has also been demonstrated for flow networks [17], limiting the possible losses that occur if agents act as price-anticipators.

The convergence properties of the lump-sum market are not well understood, but this paper presents one set of preliminary results in this area. The two-player game for agents with linear utilities is considered in detail, and convergence properties are presented for continuous steepest ascent dynamics and for discrete best response dynamics. This investigation reveals marked differences between the convergence properties of the continuous and discrete dynamics, a fact that is not immediately obvious, as the discrete dynamics are not merely defined as discretized versions of the continuous dynamic (for which it is well understood that step size adversely affects convergence). Rather, the discrete dynamic is defined as a direct update to an agent's optimal bid, or best response, if all other agents are held at their current bids. As the difference in the marginal utility of agents increases, convergence of this dynamic fails due to the fact that small changes in one bid result in large changes in the other agent's best response.

For the air traffic control problem, these results present a method for improving the convergence of lump-sum markets with multiple airlines, arbitrary convex cost functions, and numerous resources. If iterative best response bidding shows oscillatory pricing behavior in a specific situation, it is likely that fixing a maximum bid change between rounds will improve convergence, by mimicking the effect of requiring continuous dynamic updates. In practice, iterative best response bidding between price-anticipating airlines was observed to converge quite quickly, as is demonstrated in a flow control scenario presented in Section V. With eight airports, four airlines, and a 3-h time horizon, the scenario is indicative of the problem size that can be solved in real time on a standard desktop computer. The mechanism can also be scaled to manage larger networks if distributed computation platforms are considered for each of the airlines.

Market mechanisms exhibit known deficiencies when implemented for engineering applications. The mechanisms do not consider the effect of repeated daily operation of the market on the airline strategies, nor do they consider the possibility that airlines might take into account the multiple stages of the bidding process. It will be necessary to ensure that price fluctuations do not result in unsafe situations, although this concern is mitigated by the 3-h planning horizon. There are also issues relating to the adoption of a novel resource allocation method which represents a paradigm shift in U.S. air traffic management, as most en route resources are currently free of charge and airport fees are not slot dependent. Despite these open issues, this paper presents a first step toward obtaining cost information from strategic airlines and incorporating it into air traffic control decision-making. Resolving these deficiencies for resource allocation in air traffic control remains an area of future work.

This paper proceeds as follows. Section II defines a single resource lump-sum market and restates the main known results concerning existence and efficiency of a Nash equilibrium. The convergence properties of the lump-sum market for two agents with linear utilities are then investigated in Section III, for both iterative best response play and continuous steepest ascent dynamics. In Section IV, the air traffic network flow resource allocation problem is restated for completeness, as first defined in [7], and the lump-sum mechanism definition is extended to the network flow problem, as described in [17]. Simulation results are presented for an air traffic network consisting of eight airports in the northeastern United States over a 3-h horizon with inclement weather affecting the destination airport in New York. Finally, some of the difficulties inherent in implementing the proposed mechanism and areas of future work are described in Section VI.

SECTION II

## LUMP-SUM MARKET MECHANISM

### A. Lump-Sum Market for a Single Resource

Consider a system of a single resource with constrained quantity . Competing for the resource are agents who value their allocation according to a utility function . The agent utility functions are assumed to be both increasing and concave over the compact space defined by *X*_{j} = [0,*C*]. A centralized formulation of the efficient allocation problem is defined in Problem P2.1.
TeX Source
$$\matrix{\noalign{\hrule\vskip 2.5pt}\smash{\vrule depth 43.5pt height 42pt}&\eqalign{&\hbox{ Central Allocation Problem}\cr\noalign{\vskip 3pt} &{\hskip 20pt}\hbox{maximize}\quad{\hskip 1pt}\sum\limits_{j\in{\cal J}}\mathtilde{U}_{j}(x_{j})\cr\noalign{\vskip 4pt}&{\hskip 20pt}\hbox{subject to}\quad\sum\limits_{j\in{\cal J}}x_{j}\leq C}\qquad{\hbox{(P2.1)}}&\smash{\vrule depth 43.5pt height 42pt}\cr\noalign{\vskip 9pt\hrule}}$$This convex program can readily be solved if all utility information is known, but, for systems with independent agents and a competitive environment, it is improbable that this information will be available to a central authority. Instead, market mechanisms can be employed to reach an agreement on resource allocation amongst agents without requiring full disclosure of utility functions.

A lump-sum market is defined in which each agent specifies a lump-sum payment for the resource. This results in an elastic demand that balances demand with capacity using the pricing rule
TeX Source
$$\lambda={\sum w_{j}\over C}.\eqno{\hbox{(1)}}$$The resource price is defined as and the notation ∑ *w*_{j} is used when summing over all agents. An agent's lump-sum bid results in the allocation
TeX Source
$$x_{j}={w_{j}\over\sum w_{j}}C.\eqno{\hbox{(2)}}$$

In abstracting resource markets for analysis, it is essential to explicitly define models of agent information and behavior that accurately capture the likely actions of agents in the real system. For markets with large numbers of similarly sized participants, it is reasonable to assume that agents assume prices are beyond their influence. Such agents are said to be *price-taking* and operate rationally with respect to the fact that they believe their bids have no effect on the market prices they experience. Restricting the price to be strictly positive (i.e., requiring that the resource is contested), a price-taking agent in a lump-sum market optimizes the following local utility
TeX Source
$$U_{j}^{PT}(w_{j};\lambda)=U_{j}\left({w_{j}\over\lambda}\right)-w_{j}.\eqno{\hbox{(3)}}$$The notation for the arguments of the utility function *U*_{j}^{PT}(*w*_{j};λ) employs a semicolon to distinguish between arguments that are controlled by agent *j*, such as its bid, and those that are assumed by agent *j* to be determined exogenously, such as the price.

For markets with only a few participants, however, it is unreasonable to assume such naïvety on the part of competitive agents. Since the pricing rule for lump-sum markets can be assumed to be publicly available, it is likely that agents will anticipate the effect of updating their bids on market prices. This behavior is referred to as *price-anticipating*. A modified price-anticipating agent utility function is defined that incorporates the resource pricing rule and is expanded to account for the case in which all bids are zero
TeX Source
$$U_{j}^{\rm PA}(w_{j};w_{-j})=\cases{U_{j}\left({w_{j}\over\sum w_{j}}C\right)-w_{j},&if $w_{j}>0$\cr U_{j}(0),&otherwise.}\eqno{\hbox{(4)}}$$For notational simplicity, the term *w*_{−j} refers to the set of all bids other than agent *j*'s.

The appropriate equilibrium condition for a market with price-anticipating agents is that of the Nash equilibrium, where each agent prefers its current bid to any other, given that all other agents hold their bids fixed. A *Nash equilibrium* is defined as a vector of bids *w*^{NE}≥ 0 such that
TeX Source
$$U_{j}^{\rm PA}\left(w_{j}^{\rm NE};w_{-j}^{\rm NE}\right)\geq U_{j}^{\rm PA}\left(w_{j};w_{-j}^{\rm NE}\right),\quad\forall w_{j}\in\BBR_{+}.\eqno{\hbox{(5)}}$$

### B. Equilibrium Properties for a Single Resource

Two key results in the analysis of lump-sum markets with price-anticipating agents are restated here, which establish existence and uniqueness of the Nash equilibrium as well as a bounded efficiency loss relative to the optimal solution to the central allocation problem of no more than 25%.

The following theorem shows that a solution to the game exists in the form of a unique Nash equilibrium.

#### Theorem 1 (Existence, Uniqueness [18])

*Assume that for **J* > 1, each agent *j* has a concave, increasing, and continuously differentiable utility function *U*_{j}. Then there exists a unique Nash equilibrium *w*^{NE}≥ 0 that is the unique solution to the modified central allocation problem
TeX Source
$$\matrix{\noalign{\hrule\vskip 2.5pt}\smash{\vrule depth 43.5pt height 42pt}&\eqalign{&\hbox{Modified Central Allocation Problem}\cr\noalign{\vskip 3pt} &{\hskip 20pt}\hbox{maximize}\quad{\hskip 1pt}\sum\limits_{j\in{\cal J}}\mathtilde{U}_{j}(x_{j})\cr\noalign{\vskip 4pt}&{\hskip 20pt}\hbox{subject to}\quad\sum\limits_{j\in{\cal J}}x_{j}\leq C}\qquad{\hbox{(P2.2)}}&\smash{\vrule depth 43.5pt height 42pt}\cr\noalign{\vskip 9pt\hrule}}$$where
TeX Source
$$\kern-12pt\mathtilde{U}_{j}(x_{j})\!=\!\left(1-{x_{j}\over C}\right)U_{j}(x_{j})+{x_{j}\over C}\!\left({1\over x_{j}}\int\limits_{0}^{x_{j}}\!U_{j}(y)dy\!\right)\!,\ \ \forall j\in{\cal J}\eqno{\hbox{(6)}}$$*and the allocation is defined by (2)*.

Even though the modified utility function does not have a clear interpretation, it is a useful construct in that it enables the proof of existence and uniqueness of the Nash equilibrium. The following theorem ensures that efficiency loss in the game is bounded.

#### Theorem 2 (Efficiency Loss [17])

*Let **x*^{S} be the unique solution to the central allocation problem and *x*^{G} be the unique solution to the game (*U*_{1}^{PA},…,*U*_{J}^{PA}) and hence to the central allocation game. Then
TeX Source
$$\sum_{j\in{\cal J}}U_{j}\left(x_{j}^{G}\right)\geq{3\over 4}\sum_{j\in{\cal J}}U_{j}\left(x_{j}^{S}\right).\eqno{\hbox{(7)}}$$*Furthermore, for any set of agents **J*, there exist linear utilities for which this bound is tight.

This result is the main reason that lump-sum markets have recently been proposed as an alternate to resource markets for network resource allocation problems [17].

SECTION III

## CONVERGENCE PROPERTIES OF LUMP-SUM MARKETS

In order for a mechanism such as the lump-sum market to be useful in an engineering application such as air traffic control, its convergence properties must be understood. Convergence results for best response dynamics are typically difficult to achieve, although some positive results have been established [19], [20]. By focusing on the two-player game with linear utilities, it is possible to gain some insight into the convergence properties of the lump-sum mechanism in general. To this end, the utility function for each price-anticipating agent is defined as
TeX Source
$$U_{j}^{\rm PA}(w_{j};w_{-j})=c_{j}{w_{j}\over\sum w_{j}}C-w_{j},\quad j=\{1,2\}$$where *c*_{j} is agent *j*'s constant marginal rate of return for additional units of the resource. Throughout this section, the symmetry of the problem is exploited to simplify notation by assuming that *c*_{2}/*c*_{1}≥ 1, without loss of generality.

### C. Discrete Dynamics

The discrete dynamic can now be defined by setting the order in which agents respond to changes in each others' bids, be it sequentially, concurrently, or randomly. Let *k*∊ index the time step; then the concurrent discrete dynamic *w*(*k*+1) = β^{c}(*w*(*k*)) is defined for each agent as
TeX Source
$$w_{j}(k+1)=\beta_{j}^{c}\left(w(k)\right)=\beta_{j}\left(w_{-j}(k)\right),\quad\forall j\in{\cal J}.\eqno{\hbox{(13)}}$$The sequential discrete dynamic *w*(*k*+1) = β^{s}(*w*(*k*)) is defined as
TeX Source
$$\eqalignno{w_{j}(k+1)=&\,\beta_{j}^{s}\left(w(k)\right)\cr=&\,\cases{\beta_{j}\left(w_{-j}(k)\right),&if $k+1\ {\hbox{mod}}\ j=0$\cr w_{j}(k),&otherwise.}&\hbox{(14)}}$$

Examples of the sequential update for the two-player game can be seen in Fig. 3 for varying values of agent utilities. Note that as Agent 2's utility per unit resource increases relative to Agent 1's, the sequential update dynamics become less stable, resulting in limit cycle behavior in Fig. 3(c) and instability in Fig. 3(d).

*1) Local Stability*

Local stability of the equilibrium can be determined by investigating the eigenvalues of the linearized dynamics by the Hartman–Grobman theorem, stated here in its original continuous form.

#### Theorem 3 (Hartman–Grobman [21])

*If the Jacobian linearization **Df*(*w*) of the system at the equilibrium *w*∗ has no zero or purely imaginary eigenvalues, then there exists a neighborhood *U*∊^{n} and a homeomorphism *h*: *U*→^{n} that maps trajectories from the system to its linearization. In particular, *h*(*w*∗) = 0, and the homeomorphism can be chosen to preserve the parameterization by time.

As a result, stability of the equilibrium with linearized dynamics applies to the nonlinear dynamics over some local region about the equilibrium. If the linearized dynamics have eigenvalues in the left half-plane, the nonlinear dynamics are locally exponentially stable for an open neighborhood of the equilibrium. Similarly, if any eigenvalue of the linearized dynamics is in the right half-plane, the nonlinear equilibrium is unstable. An analogous theorem exists for the discrete case [21], and local stability can be guaranteed by ensuring eigenvalues of the linearized discrete system are located inside the unit circle.

Starting with the concurrent discrete dynamic, the linearized dynamics about *w*∗ are
TeX Source
$$\eqalignno{\left(w(k+1)-w^{\ast}\right)=&\,D\beta^{c}(w)\vert_{w^{\ast}}\left(w(k)-w^{\ast}\right)\cr=&\,\left[\matrix{0&\displaystyle{c_{1}-c_{2}\over 2c_{2}}\cr\displaystyle{c_{2}-c_{1}\over 2c_{1}}&0}\right]\left(w(k)-w^{\ast}\right)&\hbox{(15)}}$$with eigenvalues
TeX Source
$$\lambda=\left[\matrix{\displaystyle{c_{1}-c_{2}\over 2\sqrt{-c_{2}c_{1}}}\cr\cr\displaystyle{c_{2}-c_{1}\over 2\sqrt{-c_{2}c_{1}}}}\right]\eqno{\hbox{(16)}}$$which remain inside the unit circle for ratios of . Above this ratio, the equilibrium is locally unstable. Note that the sequential update procedure results in the same linearization as the concurrent update, as can be demonstrated by considering the composition of two sequential updates.

*3) Convergence*

Depending on the ratio of utility slopes *c*_{2}/*c*_{1}, the discrete dynamics evolve in a number of manners. This section first identifies the ratios for which convergence is possible, guarantees convergence for a subset of these ratios, and then describes the behavior that results above these ratios.

The maximum bid of either agent is found by maximizing its best response function over all bids by the other agent. Setting the best response function derivative to zero for agent 1 yields
TeX Source
$$\eqalignno{{\partial\beta_{1}\over\partial w_{2}}=&\,{\sqrt{c_{1}}\over 2\sqrt{w_{2}}}-1=0\cr w_{2}^{\max}=&\,c_{1}/4\cr\beta_{1}\left(w_{2}^{\max}\right)=&\,c_{1}/4.&\hbox{(17)}}$$The maximum bid by agent 1 occurs when agent 2 bids *c*_{1}/4 and is also *c*_{1}/4. Convergence to the equilibrium is shown for all feasible initial bids *w*_{0}∊Λ = (0,*c*_{1}/4]× (0,*c*_{2}/4] for a limited range of utility slope ratios.

*Theorem 4*

*Given utility slope ratios of **c*_{2}/*c*_{1}∊[1,9/4), dynamics defined by (14), and any initial bid pair *w*_{0}∊Λ, the system converges to the equilibrium
TeX Source
$$w^{\ast}=\left({c_{1}^{2}c_{2}\over(c_{1}+c_{2})^{2}},{c_{1}c_{2}^{2}\over(c_{1}+c_{2})^{2}}\right).$$*Please refer to the Appendix for the details of the proof*.

For ratios of , it is possible to demonstrate that the best response function of agent 2, β_{2}(*w*_{1}), is positive for all *w*_{1}∊(0,*w*_{1}^{max}]. This ensures that the only games in which either agent will post a bid value of zero occur when agent 2 bids above the maximum value at which agent 1 will participate. Setting β_{1}(*w*_{2}) = 0 yields *w*_{1}^{pmax } = *c*_{1} as that maximum value, and then setting β_{2}(*w*_{1}^{max}) = *w*_{1}^{p max} and solving for the ratio *c*_{2}/*c*_{1}, gives a limit of *c*_{2}/*c*_{1} = 25/4. Therefore, for all *c*_{2}/*c*_{1} > 25/4, the discrete dynamic outcome may be undefined, and for all , the system is not able to converge to the equilibrium, as it is locally unstable, nor is it able to exit the region of feasible bids (0,*c*_{1}/4]× (0,*c*_{2}/4]. Instead, all trajectories must continue throughout the feasible region as *t* goes to infinity.

In summary, the two-player game with discrete dynamics can be subdivided into four types based on the ratio of utility slopes *c*_{2}/*c*_{1}.

For ratios of , the equilibrium is locally stable, and convergence to the equilibrium can be guaranteed for all initial values of *w*∈(0,*c*_{1}]×(0,*c*_{2}] and utility slopes *c*_{2}/*c*_{1} < 9/4.

For slopes , no guarantee is provided, although it is our conjecture that all trajectories of the discrete dynamics do converge to the equilibrium.

If the ratio of , two further possibilities exist. First, for ratios of *c*_{1} < 25/4, convergence to the equilibrium is not possible and a limit cycle occurs.

Then, for ratios *c*_{2}/*c*_{1} > 25/4, the possibility arises that one agent is forced to bid zero, at which point the best response function for the other agent is undefined, and the outcome is undefined.

Returning to Fig. 3(a)–(d), it can be seen that in scenarios (a) (b), the discrete dynamic achieves the equilibrium; in (c), a limit cycle results, and in (d), a zero bid is placed by agent 1, resulting in an undefined outcome. As expected, the ratios of *c*_{2}/*c*_{1} for which each of these behaviors occurs fall into the regions described in this section.

This result is interesting from the point of view of applying the mechanism to the air traffic flow control problem, as it demonstrates that even with only two price-anticipating airlines, it is possible that iterative best response bidding may not converge to an equilibrium, leaving controllers without a traffic-flow solution to implement. The following section performs a similar analysis for a continuous airline bidding update model with more favorable results.

### D. Continuous Dynamics

Instead of allowing an agent to bid its best response directly, it is possible to define continuous dynamics for the game whereby each agent seeks to locally improve in the direction of steepest ascent of its value functions. Additionally, we can define an aggressiveness factor for each agent κ_{j}, which sets the rate at which an agent reacts to its value function gradient. The continuous dynamic in terms of bids is defined as
TeX Source
$$\eqalignno{\mathdot{w}_{j}(t)=&\,f_{j}\left(w(t)\right)=\kappa_{j}\left({\partial\over\partial w_{j}}v_{j}\left(w_{j}(t);\sum w_{-j}(t)\right)\right) \cr=&\,\kappa_{j}\left({c_{j}\sum w_{-j}(t)\over\left(\sum w_{j}(t)\right)^{2}}-1\right) &\hbox{(18)}}$$which is well defined for all ∑ *w*_{j} > 0 and is defined as ∞ for ∑ *w*_{j} = 0. Recall also that *w*_{j}≥ 0, which introduces a discontinuity in the definition. The dynamics for the complete system are summarized as , and example trajectories are presented in Fig. 4 for the two-player game for varying levels of aggressiveness.

*1) Local Stability*

The linearized continuous dynamics are . At *w*∗ = (*c*_{1}^{2} *c*_{2}/(*c*_{1}+*c*_{2})^{2}, *c*_{1} *c*_{2}^{2}/ (*c*_{1}+*c*_{2})^{2}), the eigenvalues of the linearized system can be shown to be negative for all values of κ_{1},κ_{2},*c*_{1},*c*_{2} > 0 but are omitted for space considerations.

*2) Convergence of Continuous Dynamic*

To demonstrate convergence of the continuous dynamic, a series of definitions and theoretical results are required. This section relies on an established body of work in nonlinear analysis [21].

First, a trajectory is defined as γ(*t*,*w*_{0}), where γ: _{+}× ^{2}→^{2}, *t*∊_{+}, and *w*_{0}∊^{2}. An *invariant set* is one for which no trajectory that starts in *S* at *t*_{0} = 0 exits the set *S*. A point *w* is an ω-*limit point* of a trajectory γ if there exists a sequence of times *t*_{n} such that *t*_{n}→∞ as *n*→∞ for which lim_{n→∞}γ(*t*_{n},*w*_{0}) = *w* and can be interpreted as any point that is visited infinitely often. An ω-*limit set* ω(*w*_{0}) of point *w*_{0} is the set of all ω-limit points of the trajectory γ(*t*,*w*_{0}). *Closed orbits* are defined as periodic trajectories that do not contain any equilibria. Finally, *heteroclinic* (respectively, *homoclinic*) orbits are defined as the union of multiple equilibria (respectively, a single equilibrium) and trajectories connecting them.

To complete this brief overview of the necessary concepts, two well-known results are restated. The first, known as Andronov's theorem, further characterizes ω-limit points.

#### Theorem 5 (Andronov's Theorem [22])

*Every ω-limit point must be an equilibrium, an element of a closed orbit, or an element of a heteroclinic or homoclinic orbit*.

Next, Bendixson's theorem can be used to determine the lack of existence of closed orbits inside a simply connected region.

#### Theorem 6 (Bendixson's Theorem [21])

*Suppose is a simply connected region such that the divergence **div*(*f*) = (∂ *f*_{1}/∂ *w*_{1})+(∂ *f*_{2}/∂ *w*_{2}) is not identically zero in *D* and does not change sign anywhere in *D*. Then *D* contains no closed orbits of .

It is possible to demonstrate, in three steps, convergence of the continuous dynamic defined in Section II-A over a set of initial conditions. The first step is to demonstrate that the set of feasible initial conditions defines an invariant set for any values of κ_{1},κ_{2} and for values of *c*_{2}/*c*_{1} < 25/4. The second step involves showing that closed orbits do not exist inside the region of interest, which is done using Bendixson's theorem for planar systems. Finally, it is possible to demonstrate that the equilibrium is the unique element of the ω-limit point for all points inside the invariant set. By Andronov's theorem, it is known that all ω-limit points must be equilibria, closed orbits, or heteroclinic/homoclinic orbits. Closed orbits can be eliminated by use of Bendixson's theorem, heteroclinic orbits by the fact that only one equilibrium exists inside the invariant region, and homoclinic orbits by the local stability of the equilibrium. This guarantees that all trajectories must eventually reach the equilibrium. Since the equilibrium is locally stable, a trajectory cannot leave the equilibrium once it has been reached, and convergence is ensured.

An invariant set for the continuous dynamics can be defined by the set of feasible initial conditions, which is all points where both agents bid no more than the maximum of their best response functions (*w*_{1},*w*_{2})∊[0,*c*_{1}/4]× [0,*c*_{2}/4]. For cases where *c*_{2}/*c*_{1} > 4, the region can be further restricted to ensure that neither agent is priced out of the market on the initial bid, namely, that *w*_{2}∊[0,*c*_{1}]. The region under consideration can be defined by four separate curves, as depicted in Fig. 5, labeled Ω ,and defined in Table 1.

*Lemma 1*

The region Ω defined in Table 1 is an invariant set for the system defined in (18).

Please refer to the Appendix for details of the proof.

It is now possible to state the main convergence result for continuous dynamics with two agents.

*Theorem 7 (Convergence to Nash Equilibrium)*

*Given utility slope ratios of **c*_{2}/*c*_{1}∊[4/25,25/4], dynamics defined by (18), and any initial bid pair *w*_{0}∊Ω, the system converges to the equilibrium
TeX Source
$$w^{\ast}=\left({c_{1}^{2}c_{2}\over(c_{1}+c_{2})^{2}},{c_{1}c_{2}^{2}\over(c_{1}+c_{2})^{2}}\right).$$*Please refer to the Appendix for details of the proof*.

Although this result is limited to utility slope ratios *c*_{2}/*c*_{1} < 25/4, it is our conjecture that convergence holds beyond this threshold as well. The difficulty in extending the result lies in the discontinuity in the definition of the continuous dynamics when a zero bid is placed by either agent in reaction to a bid placed by one agent above the maximum at which the other agent will participate, which eliminates the ability to apply either Bendixson's or Andronov's theorem.

The analysis of convergence properties in this section reveals that the continuous update dynamics are stable for a much larger range of agents than iterative best response bidding. This result is in contrast with the desire of mechanism designers to allow the more natural mechanism definition where agents bid iteratively and react to current market prices. The lesson for the air traffic control application is therefore that if issues arise with limit cycles or zero bids in the iterative setting, convergence to the unique Nash equilibrium can be improved by approximating continuous ascent dynamics. In practice, this can be done by enforcing a maximum bid adjustment step size.

The methods employed for the convergence analysis rely heavily on the fact that only two agents were considered. In extending these results to more than two agents that exist for air traffic control, a novel approach such as a Lyapunov stability argument would most likely be needed. Nonetheless, the insight provided by the two agent scenario should be applicable to problems with more agents.

SECTION IV

## APPLICATION TO AIR TRAFFIC FLOW CONTROL

The lump-sum market mechanism described above can be seen to satisfy three crucial properties required for the inclusion of airline preference information in the air traffic flow control problem. First, an equilibrium solution is known to exist even when airlines anticipate the effect of their bids on prices. Secondly, this equilibrium is known to exhibit a worst case loss in efficiency of 25%, ensuring that strategic airline behavior will not result in arbitrarily poor system performance. Thirdly, the convergence results from the previous sections reveal that it is possible to achieve this equilibrium through a distributed bidding process. Although it is true that other properties for this mechanism can also be considered essential, these remain in the realm of future work.

This mechanism is applied to an air traffic network flow model first defined in [7]. The model aggregates flow separately for each airline between origin and destination pairs and seeks to avoid excessive densities of aircraft at any one resource (jetway or airport runway) at any specific point in the planning horizon, based on resource capacity estimates derived from weather forecasts. The flow control approach does not develop trajectories for individual aircraft but rather determines appropriate traffic levels for individual links to meet regional goals, such as rerouting traffic flows around congestion or weather. This approach is reasonable in light of the hierarchical structure of the NAS, where more localized tasks of aircraft separation assurance are handled by TRACON and sector controllers, while regional concerns are in the realm of traffic-flow management at the center level. Also, since aircraft operate most efficiently at their desired cruising speed, and since the flow abstraction necessarily reduces the model fidelity, the flows are assumed to operate at fixed velocities, which greatly reduces the computational complexity of the flow allocation problem.

It is important to note that other models exist for the NAS resource allocation problem [6], [23], which enable variable velocity flow modeling, or tracking of individual aircraft. This specific flow model was selected, however, for its convex and scalable formulation, which makes it ideally suited for use with the lump-sum market mechanism and for application to large portions of the NAS. This section proceeds by first presenting an overview of the preferred flow modeling approach. The lump-sum mechanism is then adapted to the network flow allocation setting, following its definition in [17], which is followed by a discussion of the computational complexity of the optimization required of the airlines. The mechanism is then implemented on a multiairport scenario in Section V.

### A. Network Definition

Consider an air traffic network defined by fixed sets of nodes *n*∊{1,⋖,*N*} = *N* and links *i*∊{1,…,*I*} = *I*. The node incidence matrix *A*^{N}∊{0,1}^{N×N} captures the connectivity of the network, with *A*_{ab}^{N} = 1 if node *a* is connected to node *b*, and zero otherwise. Similarly, a link connectivity matrix *A*^{I}∊{0,1}^{I× I} is defined with *A*_{ab}^{I} = 1 if link *a* flows into link *b*, and *A*_{a,b}^{I} = 0 otherwise.

A subset of links is defined as the set of sources from which flow can originate and is defined as the set of sinks from which flow exits the network. Each link *i*∊ *I* has a set of links *M*_{i}, whose entire flow merges into link *i*; and a set of links *D*_{i}, whose members each receive a portion of flow from link *i*. Let *M* = ∪_{i∊ I} *M*_{i} represent the set of all merging links and define *D* similarly. Then, for each diverging link *d*∊ *D*, there exists a nonempty set *F*_{d} of inflow links for which a portion of the flow is able to flow into link *d*. See Fig. 6 for a small example of the elements of this common network definition.

### B. Discrete Path Flow Model

Instead of allowing complete authority over flow velocity and route selection, the path flow model restricts the flow to predefined paths with fixed velocity profiles. The result is that continuity is conserved by the path definition and that capacity constraints need only be ensured upon entry to a link. The formulation is further simplified by discretizing time with time step δ *t*, as *t*∊{0,δ *t*, …,*T*} = *T*, where *T*∊_{+} represents the finite time horizon of interest and *N*_{T} = | *T*|. This allows for the definition of a finite set of available link/time pairs referred to as resources *r* = (*i*,*t*)∊ *R* = *I*× *T*, with *R* = | *R*| the total number of resources. Each resource has associated with it an en route capacity limit, summarized by an en route resource capacity limit vector *C*^{e}∊_{+}^{R}. Once again, the assumption that these limits can be determined safely and in advance is implicit in the definition of the capacity of resources.

In the Discrete Path Flow model, airlines plan flights for specific origin–destination (OD) pairs (*n*_{o},*n*_{d})∊ . Each flight must follow a route over the network, which can be started at any point in the planning horizon (i.e., delayed arbitrarily). A route/time pair is referred to as a path *p*, which can be represented in terms of the resources the flights will consume, and implicitly defines the speed at which the aircraft travel over the network *p* = (*r*^{1},…,*r*^{Np}), where *N*_{p}∊ is the length of the path. Note that each path is defined for a specific airline,^{2} so that *p*∊ *P*_{j}, , and *P* = |; *P*|. Each resource is assigned to the path at the time of entry onto a link, and therefore encapsulates the dynamics of the network flow in a matrix representation. The path-resource matrix *A*^{e}∊{0,1}^{R× P} is defined as
TeX Source
$$A^{e}(r,p)=\cases{1,&path $p$ consumes resource $r$\cr 0,&otherwise.}\eqno{\hbox{(19)}}$$Traffic flow along each path can now be allocated such that capacity limitations are observed. Define *y*∊_{+}^{P} as the flow allocation along all paths; then en route capacity limit constraints are *A*^{e} *y*≤ *C*^{e}.

Similarly, airport departure and arrival flow can be restricted to satisfy runway capacity limitations. Since multiple routes can be defined for each OD pair, airport arrival and departure flow can be determined by summing over the corresponding path flow allocations. Define the matrices Ψ^{o},Ψ^{d},∊{0,1}^{V× P} to map any path flow allocation *y* to origin–destination pair inflows *y*^{o}∊_{+}^{V}, and outflows *y*^{d}∊_{+}^{V}, where *V* = *OD* × *T* is the set of OD/time pairs, of cardinality *V*. Then
TeX Source
$$\Psi^{o}(v,p)=\,\cases{1,&path $p$ starts at OD/time pair $v$\cr 0,&otherwise}\eqno{\hbox{(20)}}$$*y*^{o} = Ψ^{o} *y* and *y*^{d} is defined similarly. The airport departure and arrival flow can be defined by summing over all OD pairs that arrive at or depart from a specific airport. Similar to (19), let be the origin and destination matrices, respectively. Then airport capacities are constrained by *A*^{o} *y*^{o}≤ *C*^{o} and *A*^{d} *y*^{d}≤ *C*^{d}, where *C*^{o}^{|||S||× NT} and *C*^{d}^{|||Z||× NT} are the limits on safe airport operation.

### C. Discrete Airline Model

In order to react strategically to unpredictable phenomena such as weather, it is necessary to define an airline cost function over possible outcomes. For airlines, aircraft arrival delays relative to the predefined flight schedule are an excellent indicator of the costs to be incurred by air traffic control decisions, and are in fact monitored by the FAA to assess air traffic control performance [1]. Therefore, the airline costs are modeled by minimizing the quadratic deviation from the cumulative scheduled arrival flow, which penalizes not only deviations from the scheduled flow but also how long those deviations persist. This cost structure does not exactly penalize individual flight delays, however, as this information is lost in the flow aggregation process. The quadratic nature of the cost function does penalize large deviations from the flight schedule more heavily than small ones, which mimics the fact that larger disruptions tend to impact subsequent scheduled flights more severely, driving a domino effect of cost escalation as delays persist. Although deviations from the planned routes, departure delays, and en route velocity changes can impact airline costs, it is reasonable to assume that arrival delays are the most prominent concern. However, concave, nondecreasing cost functions can easily be designed that also penalize additional quantities such as cumulative deviation from the scheduled departure flow or deviation from scheduled path flows. Future work will aim to include the case of airline flight cancellations as well, which essentially define an outside option as resource prices increase.

To simplify notation, let *Y*^{o}∊_{+}^{V} and *Y*^{d}∊_{+}^{V} be the cumulative origin inflow and destination outflow for all OD/time pairs, respectively. Cumulative departure inflow is defined by *Y*^{o} = Γ *y*^{o}, where Γ∊{0,1}^{V× V} is a block diagonal matrix with lower triangular blocks of dimension *N*_{T}, and cumulative arrival inflow is defined similarly. Let *y*^{so} be the scheduled departure flow and *y*^{sd} be referred to as the scheduled arrival flow, with *Y*^{so} and *Y*^{sd} defined equivalently to *Y*^{o} and *Y*^{d}. A representative airline cost function can be defined as
TeX Source
$$J_{j}(y_{j})=\left\Vert\Omega_{j}\max\left({\bf 0},Y_{j}^{sd}-Y_{j}^{d}\right)\right\Vert_{2}^{2}\eqno{\hbox{(21)}}$$where Ω_{j}∊_{+}^{V× V} is a diagonal weighting matrix that captures each airline's preferences over OD pairs at each time epoch. Note that the zero vector of dimension *V* is represented by **0** and the max function is performed elementwise; that is, **max** (**0**,·) operates on a vector such that positive entries are unchanged and all other entries are set to zero. Equation (21) defines an example of an airline cost function that is convex and nonincreasing, since it is a positive linear combination of convex, nonincreasing functions in the path flow variables *y*_{j}.

### D. Centralized Optimization

It is now possible to define a centralized optimization program with complete information of airline preferences. Let *A* and *C* be defined as
TeX Source
$$A=\left[\matrix{A^{e}\cr A^{o}\cr A^{d}}\right],\quad C=\left[\matrix{C^{e}\cr C^{o}\cr C^{d}}\right].\eqno{\hbox{(22)}}$$Note that the matrix A can be divided into *N*_{J} submatrices *A* = [*A*_{1}|…| *A*_{NJ}], which map airline path flow allocations *y*_{j} to the resources consumed. The air traffic path flow allocation problem can now be seen to take a similar form to the central allocation problem defined in Problem P2.1
TeX Source
$$\matrix{\noalign{\hrule\vskip 6.5pt}\smash{\vrule depth 40pt height 42pt}&\eqalign{&\hbox{Central Path Flow Allocation}\cr\noalign{\vskip 3pt} &{\hskip 20pt}\buildrel{\hbox{maximize}}\over{\scriptstyle{{y\geq 0}}}\quad{\hskip 1pt}\sum\limits_{j\in{\cal J}}J_{j}\cr\noalign{\vskip 4pt}&{\hskip 20pt}\hbox{subject to}\quad Ay\leq C}\qquad{\hbox{(P4.1)}}&\smash{\vrule depth 40pt height 42pt}\cr\noalign{\vskip 9pt\hrule}}$$whose solution *y*^{c}, represents the ideal solution to the multiairline resource allocation problem if airline preferences were public information. The convex formulation ensures that for problems of reasonable size, the optimization can be performed conveniently using standard convex optimization tools such as CVX and SeDuMi in the Matlab environment [24], [25], [26].

### E. Lump-Sum Market Mechanism for Air Traffic Flow Control

The definition of the lump-sum market for this problem formulation requires additional information of the bidders, in order to avoid situations where a Nash equilibrium might not exist if demand for a resource is insufficient to keep prices away from zero [17]. Bidders are also required to submit a link flow request , which is the amount of flow they would like to have allocated when the price for a resource is zero. Let be defined as the *airline strategy,* with *z* = (*z*_{1},…,*z*_{J}) the complete strategy vector. The flow allocation process is then augmented for each resource as follows:
TeX Source
$$x_{ij}(z)=\cases{{w_{ij}\over\sum\limits_{j\in{\cal J}}w_{ij}}C_{i}&$\sum\limits_{j\in{\cal J}}w_{ij}>0$\cr\mathhat{x}_{ij}&$\sum\limits_{j\in{\cal J}}w_{ij}=0,\sum\limits_{j\in{\cal J}}\mathhat{x}_{ij}\leq C_{i}$\cr 0&$\sum\limits_{j\in{\cal J}}w_{ij}=0,\sum\limits_{j\in{\cal J}}\mathhat{x}_{ij}>C_{i}$}\eqno{\hbox{(23)}}$$where *x*_{ij}(*z*) is the allocation of flow on link *i* assigned to airline *j* resulting from the airline strategy *z*. Note that if the total requests for flow on a link exceed the link capacity but the sum over all bids on that link is zero, no airline receives any flow allocation as the link has become contested.

The local airline cost functions are defined as
TeX Source
$$J_{j}^{\rm PA}(z_{j};z_{-j})=J_{j}\left(y_{j}\left(x_{j}(z)\right)\right)+\sum_{i\in{\cal I}}w_{ij}.\eqno{\hbox{(24)}}$$Note that the airlines not only determine the lump sum bids, *w*_{j}, to place for each resource, but also identify the path flows, *y*_{j}, which maximize the value of the flow allocations over each of the links, *x*_{j}, which are determined by the lump sum bids. In effect, the requirement to identify path flows in addition to resource bids represents an embedded network flow maximization problem in the argument of the airline cost.

The definition of a Nash equilibrium of (5) is augmented to include the flow requests
TeX Source
$$J_{j}^{\rm PA}\left(z_{j}^{\rm NE};z_{-j}^{\rm NE}\right)\leq J_{j}^{\rm PA}\left(z_{j};z_{-j}^{\rm NE}\right),\quad\forall z_{j}\in\BBR_{+}.\eqno{\hbox{(25)}}$$

The existence of a Nash equilibrium and its efficiency loss can now be established for the multilink network problem. Because the strategy space of the airlines has been expanded to include link flow requests, the requirements on the costs can be relaxed to no longer require decreasing, differentiable costs, although the uniqueness of an equilibrium can no longer be guaranteed. In establishing these results for airlines with costs instead of utilities, the following assumption is required.

*Assumption 1*

The airline cost functions *J*_{j} are assumed to be convex, nonincreasing, and continuous. Furthermore, the cost of a null allocation for each resource is assumed to be finite, and is referred as the worst case cost, *d*_{j} = *J*_{j}(0).

As a result of the worst case cost function, the airline cost functions can be posed as utility functions *U*_{j}(*x*_{j}(*z*)) = *d*_{j}−*J*_{j}(*x*_{j}(*z*)), which are nonnegative, concave, continuous, and nondecreasing for all flow allocations, and evaluate to *U*_{j}(0) = 0 for a null allocation. This assumption allows for a modest generalization of the results of [17] to problems in which airlines measure preference in terms of cost, and for which null allocations do not result in zero costs/utilities.

#### Theorem 8 (Existence [17])

*Assume that for **J* > 1, each airline *j* has a cost function *J*_{j} that satisfies Assumption 1. Then there exists a Nash equilibrium *z*^{NE}≥ 0 of the game defined by (*J*_{1}^{PA},…,*J*_{J}^{PA}) if the allocation is defined by (23).

The efficiency loss bound for lump-sum markets also holds in the current context.

#### Theorem 9 (Efficiency Loss [17])

*Let **z*^{S} be the unique efficient solution to the central air traffic control allocation problem and *z*^{G} be a Nash equilibrium of the game (*J*_{1}^{PA},…,*J*_{J}^{PA}). Then
TeX Source
$$\sum_{j\in{\cal J}}d_{j}-J_{j}\left(x_{j}^{G}(z^{G})\right)\geq{3\over 4}\left(\sum_{j\in{\cal J}}d_{j}-J_{j}\left(x_{j}^{S}(z^{S})\right)\right).\eqno{\hbox{(26)}}$$*Furthermore, for any set of airlines **J*, there exist linear utilities for which this bound is tight.

The local airline optimizations can now be defined for both price-taking and price-anticipating airlines. For airlines that act as price-takers, the airline subproblems are defined as
TeX Source
$${\hskip-2pt}\matrix{\noalign{\hrule \vskip 8.5pt}{\hskip -0.01pt}\smash{\vrule depth 37pt height 41.5pt}&{\hskip -5pt}\eqalign{&\hbox{Price-Taking Airline Subproblem}\cr\noalign{\vskip 3pt} &{\hskip 10pt}\hbox{minimize}\quad{\hskip 1pt}J_{j}(y_{j})+\sum\limits_{i\in{\cal I}}w_{ij}\cr\noalign{\vskip 4pt}&{\hskip 10pt}\hbox{subject to}\quad \lambda_{i}A_{i,j}y_{j}\leq w_{ij},\quad\forall i\in \cal I.}\qquad{\hbox{(P4.2)}}&{\hskip -11.5pt}{\hskip 11.1pt}\smash{\vrule depth 37pt height 41.5pt}\cr\noalign{\vskip 9pt\hrule}}$$The notation *A*_{i,j} denotes the *i*th row of the *j*th sub-matrix, *A*_{j}, of the matrix *A* defined in (22). *A*_{i,j} is a row vector of the same length as *y*_{j}, the path flow column vector.

For airlines that act as price-anticipators, a simplifying assumption is made that there exists a minimium bid, ∊_{w} > 0, for every resource, ensuring that only the first pricing rule in (23) is applicable. The airline subproblems are defined as
TeX Source
$$\matrix{\noalign{\hrule\vskip 6.5pt}\smash{\vrule depth 52pt height 54pt}&\eqalign{&\hbox{Price-Anticipating Airline Subproblem}\cr\noalign{\vskip 3pt} &{\hskip 10pt}\hbox{minimize}\quad{\hskip 1pt}J_{j}(y_{j})+\sum\limits_{i\in{\cal I}}w_{ij}\cr\noalign{\vskip 4pt}&{\hskip 10pt}\hbox{subject to}\quad A_{i,j}y_{j}\leq C_{i}\left({w_{ij}\over \sum\limits_{j\in \cal J} w_{ij}}\right),\quad\forall i\in \cal I.}\quad{\hskip-14pt}{\hbox{(P4.3)}}&\smash{\vrule depth 52pt height 54pt}\cr\noalign{\vskip 9pt\hrule}}$$In both cases, the airline subproblem is a convex optimization program and can therefore be solved efficiently with standard optimization tools. The airline cost functions are convex, and for Problem P4.2, the constraints are linear. The flow constraints in Problem P4.3 are also convex, which can be seen by rewriting them as
TeX Source
$$A_{i,j}y_{ij} \leq C_{i}\left(1-{\sum_{k\neq j}w_{ik}\over {w_{ij}+\sum_{k\neq j}w_{ik}}}\right)\eqno{\hbox{(27)}}$$and noting that for constant *a* ∊ (0, ∞), *w* ∊ (0, ∞), the function *f*(*w*) = *a*/(*w*+*a*) is convex in *w*.

The lump-sum market mechanism is adapted for the air traffic control problem in Algorithm 1, where iterations are indexed by *k*.

**Algorithm 1** Discrete dynamic algorithm for lump-sum markets

1: Initial bids, *z*_{j}^{0}, submitted .

2: *k* = 0

3: **repeat**

4: λ^{k} = ∑ *w*_{j}^{k}/*C*, central price update.

5: Allocations *x*_{j}^{k}(*z*)) are set as defined in (23).

6: Agents update their responses *z*_{j}^{k+1} by solving Problem P4.2 or P4.3.

7: **until** ∑‖ *z*_{j}^{k+1}−*z*_{j}^{k}‖ < ∊_{z}

### F. Problem Size

The size of the air traffic flow control problem defined above depends primarily on the length of the planning horizon and the subset of the U.S. NAS to be included. The U.S. NAS contains 60 major airports^{3} for which flow scheduling might be considered, which results in as many as 3540 OD pairs. The total number of link resources can be approximated to be 2500 by assuming approximately seven links across each of 400 sectors that cover the United States. For a path flow model with five alternate routes per OD pair and 5 min time intervals, the flow allocation problem could reach sizes on the order of one million flow variables and one-half million resource constraints. At this size, exact second-order techniques are currently impractical, and convex program solution times in the tens of minutes for each local optimization are likely on standard computing hardware. More tractable problems can be generated by including only those resources for which congestion or weather have resulted in a desired schedule that currently strains capacity limits. Similarly, the problem size is significantly improved by limiting the planning time horizon to three hours, which is in line with the time horizon for which sufficiently accurate weather predictions are available. The simulation results presented in Section V are representative of problem sizes for which real-time market-based allocation converges in a realistic time-frame for mitigating schedule disruptions due to weather.

SECTION V

## SIMULATION: AIR TRAFFIC FLOW CONTROL

### A. Scenario

A scenario based on the northeastern United States is used to investigate the relative performance of price-taking and price-anticipating airlines in the lump-sum market mechanism. For this scenario, flow departing from seven airports within approximately three hours of and destined for New York is modeled, as depicted in Fig. 7(a). Once again, the simulation spans a 3-h time horizon and uses a 5-min time step. The nodes of the network are based on actual airport and airway intersection locations available from the FAA,^{4} links were generated artificially for convenience only, and a primary shortest path and secondary alternate path were generated over the network for each OD pair.

Four airlines are considered for this scenario, as summarized in Table 2. They are defined to represent the various types of airlines in existence, including low-cost and mainline carriers, as well as large- and small-volume airlines. The schedule definitions for mainline carriers tend to be peaked, meaning that many flights are scheduled to arrive at their hub airports at the same time, putting a strain on the airport and increasing their cost of operations. In contrast, low-cost airlines try to maintain flat schedules that utilize gates and surface crews evenly, thereby reducing their costs.

En route capacity limitations are uniformly set to 90% of the peak scheduled flow along the busiest link, and inclement weather is included as a polygon with fixed size and a known velocity that reasonably approximates standard weather pattern movements. Capacity is assumed to be restricted by 50% for all flows with any subsection of the link inside the weather polygon at each time step.

### B. Results

Fig. 7(b)–(d) shows the resulting aggregate flow over the network for all airlines at a sequence of times. The weather polygon and affected nodes are also shown, and link width is again used to represent flow volume along a link. The weather disruption polygon and affected links and nodes are represented in red (dark gray).

Fig. 7(b) shows the relatively low-demand starting period of the simulation, where most flows are taking the shortest path routes to the destination and none of the flow is close to the link capacity limits. Once again, weather has a significant impact on flow to the destination; in particular, Fig. 7(d) shows a surge of recovery flow once the weather has passed.

Fig. 8 presents the arrival traffic and pricing throughout the simulation for both price-taking and price-anticipating airlines, as well as the resulting flow and constraint satisfaction. From this figure, one can see the flow at the destination airport being reduced during the weather disturbance, then recovering once the disturbance has passed. Note that the recovery does not occur immediately, as not only the destination airport but also the links leading to the destination were affected by the weather disruption, so recovery was stalled until flows along the final legs of the flight paths could again reach the airport.

The main distinction in the solutions of the price-taking airlines and the price-anticipating airlines is that the prices that result in the price-anticipating solution are lower. This effect is caused by the airlines that command a significant portion of the available resources, which bid lower than they otherwise would in order to keep prices down. In the extreme with two price-anticipating airlines and a single resource, if one airline bids zero, the other is incentivized to bid as close to zero as possible, as it will still receive all the available resource regardless of the magnitude of its bid. By contrast, the price-taking airline would bid such that the price equals the negative of its marginal cost when it receives all the available resource, which is some finite nonnegative value for strictly increasing, convex cost functions.

The deviation from the desired cumulative schedule is plotted for each airline in Fig. 9, for both the price-taking and price-anticipating airlines. The high-cost high-volume airline (Airline 1) has increased its deviations in the price-anticipating solution, whereas the other airlines (Airlines 2–4) have decreased their deviation. Essentially, the high-cost high-volume carrier commands enough of the available resources to depress market prices to its advantage, and so incurs some additional delay as a result.

A comparison of price-taking and price-anticipating airline costs is presented in Table 3, where columns represent the actual delay cost incurred, the resulting payments made by the airlines, and the total market cost that results for each airline. Three interesting elements of the results are visible in this table. First, the overall efficiency of the solution is better with price-taking airlines versus price-anticipating airlines, as visible from the total delay cost for each solution. This follows from the fact that the lump-sum market with price-taking airlines converges to the efficient solution, whereas the Nash equilibrium for price-anticipating airlines need not be efficient. Secondly, since the resource prices are lower with price-anticipating airlines, so too are the payments they make (for all but Airline 4). Lastly, all airlines prefer their market totals in the price-anticipating scenario over those in the price-taking scenario.

Finally, the market mechanism convergence performance can be observed in Fig. 10. The slow rate of convergence of the market mechanism with price-taking airlines is a direct result of the pricing update mechanism, which must maintain a small step-size to avoid excessive oscillation of market prices. Subgradient techniques can be used for the price update, but the algorithm can only be guaranteed to converge in an unbounded number of steps [27]. In contrast, with price-anticipating airlines, there is a significant improvement in the number of iterations required for the market prices to converge, which is indicative of the anticipatory effect that airlines have by including the effect of their bids on market prices in their local decision-making. In practice, the local optimizations involve convex nonlinear constraints on the airline bids, and therefore take longer to perform than the equivalent price-taking optimization, so some of the benefit of the faster market convergence is lost. For a market mechanism that requires hundreds of iterations to converge, local airline optimizations cannot consume longer than 10 s if time constraints on the overall computation are to be met.

In summary, this paper proposes the use of a lump-sum market mechanism for the allocation of resources amongst price-anticipating airlines competing for air traffic network flow, in order to improve system efficiency when weather disruptions affect capacity. This method allows for the fact that airlines are in direct competition and will attempt to gain an advantage over their competitors.

This paper includes an investigation of convergence properties of the mechanism to strengthen the justification for its implementation. By restricting the class of problems to two agents with linear utilities, both discrete best response and continuous steepest descent dynamics can be classified into regions of guaranteed stability. Most interestingly, the continuous dynamic is stable over a broader range of agent types than the discrete dynamic, which implies that for systems where convergence of the discrete best response dynamic is too slow or nonexistent, a limited step-size approach may improve the situation.

The effect of price-anticipation by the airlines on the market outcome is demonstrated for a representative flow problem, and both moderate efficiency loss and with lower resource pricing result, and in fact all airlines prefer their outcome with this mechanism over the efficient market solution that results from price-taking strategies. Although no convergence guarantee exists for price-anticipating airlines on network flow problems, in practice, the convergence results were in fact significantly better when airlines predict the effect of their bids on prices.

### A. Future Work

Many fascinating aspects of the implementation of market mechanisms for disruption management in air traffic control remain. The notion that airspace resources have inherent value, and that allocation to the highest bidder is the most desirable method for managing excess demand, is certainly not universally accepted. Great care will need to be taken to address fears from players such as the general aviation community, but ultimately both safety and efficiency concerns will be advanced if busy routes and airports are preferentially reserved for larger vehicles with higher resource valuations. Similarly, safety concerns may result if convergence to feasible solutions cannot be guaranteed within the planning window, and further work is needed in defining market performance guarantees before their use in safety-critical applications can be considered. The CDM initiative provides a useful benchmark for how to appropriately implement a market mechanism with airline participation. The ration by schedule approach for GDPs, which assigns delays based on the existing flight schedule, results in an incentive for airlines to provide accurate flight information, thereby reducing the frequency of unused landing slots. Although the mechanisms are different, the lump-sum market approach proposed in this paper does not inherently give airlines resource rights implied by the existing flight schedule. It is important for future work to address this limitation of the lump-sum market theory to ensure various stakeholders have proper incentives and are satisfied with the equity of the approach to ensure participation in a coordinated flow control solution.

Specific to the market mechanism presented in this paper, it is important to note that the price-anticipating airline model does not capture every possible action that airlines might take to attempt to manipulate the allocation mechanism. Specifically, the repeated nature of the allocation problem allows many additional strategic behaviors to be executed by the airlines that are not captured in this model. Well-known negative results demonstrate that every allocation can be sustained as an equilibrium in repeated games, through strategies that punish deviation over multiple epochs. Even in the single market case, each set of bids submitted by the airlines is evaluated separately in this model, without any notion that multiple bidding rounds are required for convergence. This is a strict limitation on the strategies considered for the airlines, and may therefore not capture the true outcome if the mechanism were implemented.

From the air traffic flow modeling perspective, further issues also remain. In particular, the conversion of the equilibrium flow solution into modifications to the existing aircraft flight plans has not been addressed. The definition of the network flow problem is inherently continuous, which is a requirement of the mechanism under consideration, whereas the existence of individual flights imposes an integer constraint on modifications. The result is that the flow solution may be difficult to achieve in practice, despite the fact that airlines are required to make payments for their flow allocations. In each case, these issues represent interesting areas of future research, with broad implications not only for air traffic management but also for many other multiagent engineering applications.

## APPENDIX

First, the proof for Theorem 4 is presented.

*Proof*

First, there exists a region Λ_{0} of increasing bids, where both agents will always increase bids until Λ_{0} is exited. This region can be identified by inspecting the best response functions and noting that β_{2}(*w*_{1}) is increasing in *w*_{1} throughout Λ, while β_{1}(*w*_{2}) is increasing for *w*_{2}∊(0,*c*_{1}/4]. Therefore, any bid pair starting in the region (0,*c*_{1}/4]×(0,*c*_{1}/4] must result in a trajectory that exits Λ_{0} at some step *k*.

For the remainder of the feasible set (0,*c*_{1}/4]×(*c*_{1}/4,*c*_{2}/4], the worst case slopes in terms of the magnitude of the eigenvalues of a linearized dynamic occur at *w*_{1} = ∊ > 0, where ∂β_{1}/∂ *w*_{2}→−1/2 as ∊→ 0 and *w*_{2} = β_{2}(*c*_{1}/4), where . The resulting eigenvalues of this worst case linearization are uniformly bounded inside the unit circle for all *c*_{2}/*c*_{1} < 9/4. Therefore, all trajectories that start in the feasible set Λ converge to the equilibrium *w*∗ for ratios of *c*_{2}/*c*_{1} < 9/4 and *c*_{2}/*c*_{1} > 1.

Next, the proof for Lemma 1 is presented.

*Proof*

Invariance is demonstrated by taking the dot product of the outward normal to the boundary and the dynamics, which is shown to be every nonpositive. Along *g*_{1}
TeX Source
$$\eqalignno{(\mathdot{w}_{1},\mathdot{w}_{2})\cdot(-1,0)=&-\mathdot{w}_{1}=-\kappa_{1}\left({c_{1}\over w_{2}}-1\right)\leq 0,\ \ \cr&\forall \ w_{2}\leq c_{1}.&{\hbox{(28)}}}$$The remaining curves can be evaluated similarly.

Finally, the proof for Theorem 7 is presented.

*Proof*

The region Ω defined in Fig. 5 is a simply connected region that is invariant to the flow defined in (18) for ratios of *c*_{j}/*c*_{k}∊[1,25/4]. Therefore, any trajectory starting from a point *w*_{0}∊Ω cannot exit Ω and must spend infinite time in the region. By Andronov's theorem, every ω-limit point must be an equilibrium, an element of a closed orbit, or an element of a heteroclinic or homoclinic orbit. There exists only one equilibrium point in Ω, as shown in Section III-B, and it is stable; therefore heteroclinic and homoclinic orbits are not possible in Ω, and so all ω-limit points must be either elements of a closed orbit or the unique equilibrium. The existence of closed orbits can be ruled out using Bendixson's theorem since the divergence of the flow field is negative for all *w*∊Ω
TeX Source
$$\eqalignno{div(f)=&\,-2\left({\kappa_{1}c_{1}w_{2}\over(w_{1}+w_{2})^{3}}+{\kappa_{2}c_{2}w_{1}\over(w_{1}+w_{2})^{3}}\right)\cr<&\,\ 0,\quad\forall\, \{w_{1},w_{2}\}\in\Omega.&\hbox{(29)}}$$Therefore, the unique ω-limit point in Ω is the equilibrium, and so all trajectories starting in Ω must converge to this equilibrium. Since the equilibrium is locally stable, a trajectory cannot leave the equilibrium once it has been reached.