Hatchet: Pruning the Overgrowth in Parallel Profiles

Performance analysis is critical for eliminating scalability bottle-necks in parallel codes. There are many profiling tools that can instrument codes and gather performance data. However, analyt-ics and visualization tools that are general, easy to use, and pro-grammable are limited. In this paper, we focus on the analytics of structured profiling data, such as that obtained from calling context trees or nested region timers in code. We present a set of techniques and operations that build on the pandas data analysis library to enable analysis of parallel profiles. We have implemented these techniques in a Python-based library called Hatchet that al-lows structured data to be filtered, aggregated, and pruned. Using performance datasets obtained from profiling parallel codes, we demonstrate performing common performance analysis tasks repro-ducibly with a few lines of Hatchet code. Hatchet brings the power of modern data science tools to bear on performance analysis.