Common metrics framework for ATM performance assessment and monitoring

Across the air traffic management (ATM) discipline there is a long-standing problem of inconsistencies in assessing and monitoring system performance. Research has offered limited solutions so far, and has failed to offer an internationally acceptable performance definition methodology that provides the ability to: (1) compare the analogous performance metrics and evaluations obtained by various organizations; (2) roll-up measures to the organizational goals; and (3) comprehend the impact of a change in metric-value onto the overall performance. This paper describes the initial results of an on-going work proposed by FAA, Eurocontrol and their partners, that addresses the ATM performance measurement discrepancies. The problem is approached in a systematic manner by dissecting the ATM system into the smallest elements and processes, and by studying the International Standardization Organization (ISO) definition of quality of service (QoS) in information technology (IT). A common metric taxonomy is developed to provide for unambiguous translation of the terminology and metric specifications used by different organizations, and to facilitate developing a collection of the existing metrics to enable maintaining the connection between the identical metrics, metrics with identical definitions but different names, and metrics with identical names but different definitions. A common ATM performance framework is built upon the International Civil Aviation Organization (ICAO) concepts of required ATM system performance (RASP) and required total system performance (RTSP). Illustrated through cause-effect diagrams, the framework maintains the connectivity between different levels of performance evaluation, performance aspects, and performance measures, and enables rolling-up measures to the performance requirements and organizational goals. A common 'dashboard' for presenting the achieved or expected performance is developed using radar diagrams to clearly and effectively accommodate simultaneous display of large amount of information. As such, this methodology enables common ATM performance monitoring, operational concept validation and comparison, and target setting. The proposed framework is briefly illustrated on the safety-evaluation example, and the further research guidelines are briefly described.