Reinforcement Learning for Airport Slot Allocation: Incorporating Fairness and Trade-off Analysis

Airport Slot Allocation plays a vital role in managing airport congestion. Approaches for Airport Slot Allocation range from exact methods to searching-based methods. Due to the scalability problem of exact methods, the literature is shifting to using searching-based methods, such as heuristics and learning-based methods. While heuristics are prone to changing scenarios, learning-based methods, on the other hand, can self-evolve with updated data. Reinforcement Learning, a learning-based approach, can automate the search and explore new solutions when the search space changes. Reinforcement Learning has been studied for Airport Slot Allocation. However, few have incorporated fairness, an important criterion in resource allocation. Incorporating fairness into Reinforcement Learning formulations heavily depends on the reward design. A sparse reward design can lead to non-convergence. A dense reward design, on the contrary, may not closely resemble the objectives, minimizing the total displacement and maintaining fairness. In this paper, we study how to efficiently incorporate fairness, via the mini-max criterion, the principle of proportionality, and the constrained-based approach, into Reinforcement Learning formulations. To incorporate via the constrained-based approach, we limit the number of actions, which correspondingly depict the maximum displacement the Reinforcement Learning agent can assign to a movement. For the other two remaining fairness approaches, we represent the fairness objective along with the minimizing displacement objective in terms of reward signals either at each time step (i.e., dense rewards) or only at terminal states (i.e., spare rewards). We perform validations with the OAG data of the Hong Kong-Singapore-Bangkok hub. Results show that giving dense reward signals significantly optimizes the objective of minimizing the total displacement, from an average of 297.53 to 124.07, for the principle-of-proportionality models. The extra dense reward signals for maintaining fairness, however, fail to achieve convergence. Removing the reward signals for the minimizing displacement objective at each time step helps to maintain proportional fairness, but increases the total displacement.