Integrating flight-related information into a (Big) data lake

Flight cancellations, departure delays, congestion in taxi times and airborne holding delays are increasingly frequent problems that negatively impact the performance, fuel burn, emissions rate and customer satisfaction at major airports in the world. However, this is just a brushstroke of the future to come. The dramatic growth in the air traffic levels has become a problem of paramount importance, leading into an increased interest for enhancing the current Air Traffic Management (ATM) systems. The main objective is to being able to cope with the sustained air traffic growth under safe, economic, efficient and environmental friendly working conditions. The ADS-B (Automatic Dependent Surveillance - Broadcast) technology plays a major role in the new ATM systems, since it provides more accurate real-time positioning information than secondary radars, in spite of using a cheaper infrastructure. However, the main flaw in the use of ADS-B technology is the generation of large volumes of data, that, when merged with other flight-related information, faces important scalability issues. In this work, we start off from a previously developed data lake for the support of the full ADS-B data life-cycle in a scalable and cost-effective way, and propose a data architecture to integrate data from different providers and reconstruct flight trajectories that can ultimately be used to improve the efficiency in flight operations. This data architecture is also evaluated using a 2-week testbed which reports some interesting figures about its effectiveness.