Flexible Linear Algebra Development and Scheduling with Cholesky Factorization

Modern high performance computing environments are composed of networks of compute nodes that often contain a variety of heterogeneous compute resources, such as multicore CPUs and GPUs. One challenge faced by domain scientists ishow to efficiently use all these distributed, heterogeneous resources. Inorder to use the GPUs effectively, the workload parallelism needs to be muchgreater than the parallelism for a multicore-CPU. Additionally, effectivelyusing distributed memory nodes brings out another level of complexity where theworkload must be carefully partitioned over the nodes. In this work we areusing a lightweight runtime environment to handle many of the complexities insuch distributed, heterogeneous systems. The runtime environment usestask-superscalar concepts to enable the developer to write serial code whileproviding parallel execution. The task-programming model allows the developerto write resource-specialization code, so that each resource gets theappropriate sized workload-grain. Our task-programming abstraction enables thedeveloper to write a single algorithm that will execute efficiently across the distributed heterogeneous machine. We demonstrate the effectiveness of ourapproach with performance results for dense linear algebra applications, specifically the Cholesky factorization.