Work-Stealing, Locality-Aware Actor Scheduling

The actor programming model is gaining popularity due to the prevalence of multi-core systems along with the rising need for highly scalable and distributed applications. Frameworks such as Akka, Orleans, Pony, and C++ Actor Framework (CAF) have been developed to address these application requirements. Each framework provides a runtime system to schedule and run millions of actors, potentially on multi-socket platforms with non-uniform memory access (NUMA). However, the literature provides only limited research that studies or improves the performance of actor-based applications on NUMA systems. This paper studies the performance penalties that are imposed on actors running on a NUMA system and characterizes applications based on the actor type, behavior, and communication pattern. This information is used to identify workloads that can benefit from improved locality on a NUMA system. In addition, two locality-and NUMA-aware work-stealing schedulers are proposed and their their respective execution overhead in CAF is studied on both AMD and Intel machines. The performance of the proposed work-stealing schedulers is evaluated against the default scheduler in CAF.