Disparity between Theory & Practice: Beyond the Worst-Case Competitive Analysis

Online algorithms are used in a variety of situations such as forex trading, cache replacement, and job scheduling etc. In an online problem, the algorithm is presented with a sequence of input in a serial fashion such that the algorithm does not have knowledge about the future inputs. For instance, in case of forex, the online algorithm is presented daily exchange rates. The algorithm does not have knowledge about future exchange rates, and has to make an irreversible conversion decision on each day. Competitive Analysis is the standard tool to analyze the performance of online algorithms. Competitive analysis measures the performance of an online algorithm against a benchmark optimum offline algorithm. Competitive analysis is a worst case measure and is criticized as a pessimistic approach for performance evaluation. The assumptions of online algorithms designed under the competitive analysis paradigm also suffer from the same set of problems as competitive analysis it self. In this work, we contribute towards bridging the gap between theory and practice by considering a set of algorithms for online conversion problems and discuss the disparity between the assumed worst-case competitive ratios and experimentally achieved competitive ratios using real world data. We present modified worst-case input sequences in order to make them comparable to real world data. In addition, we also investigate, how the assumptions made by algorithms differs from real world. Further, we highlight other performance measures for online algorithms with the goal of realistic performance evaluation process.