Is Asymptotic Cost Analysis Useful in Developing Practical Parallel Algorithms

Summary form only given, as follows. Asymptotic analysis of runtime, and space, has been the cornerstone in the development of practical sequential algorithms. The analysis is not meant to predict runtimes on any particular machine, but rather to guide algorithm designers and implementors in the right direction, and let them better understand how algorithms might scale. Quicksort, Dijkstra's algorithm, dynamic programming, depth first search, for example, are fast in theory and in practice–-and scale pretty much as the theory predicts. All CS undergrads learn about these algorithms and techniques, and they are broadly implemented in many widely used libraries and applications. Over the years models have been extended to include the benefits of locality, allowing for a refined asymptotic analysis when needed. Unfortunately the jury is still out on the role of asymptotic analysis in the practice of parallel algorithms. There is no lack of theoretical work on analyzing the cost of parallel algorithms, with such work dating back almost fifty years, but these ideas have not been widely adopted in practice. There are various possible reasons, including inaccurate cost models, giant constants in the big-O, no adequate programming languages, lack of education on the topic, or perhaps parallel machines are just too complicated to model theoretically and we should just give up. In this talk I will describe how simple parallel models can be useful for developing practical parallel algorithms, and many of the theoretical ideas in parallel algorithms are useful, at least in the context of shared-memory multicore machines. I will cover work on developing practically efficient algorithms for a wide variety of applications of graphs, trees, computational geometry, and strings. As with sequential algorithms, the basic model does not account for locality, but I will discuss some simple ways to extend it.