Designing a Self-Organized Approach for Scheduling Bag-of-Tasks

This paper proposes a decentralized and self-organized agent system for dynamically load-balancing tasks arriving in the form of Bags-of-Tasks (BoTs) in large-scale decentralized systems. the approach is inspired by the emergent behavior of the sand pile model, a cellular automaton behaving at the edge of chaos. Depending on the state of the cellular automaton, rather different responses may occur when a new task is assigned to a resource. It may change nothing or generate avalanches that reconfigure the state of the system. the proportion between the abundance of avalanches and their sizes shows a power-law relation, a scale-invariant behavior that does not need to be tuned. That means that large - catastrophic - avalanches are very rare but small ones occur very often. Such a smart and emergent behavior fits well with the idea of non-clairvoyant scheduling, where tasks are load balanced into computing resources trying to maximize the performance but without assuming any knowledge on the tasks features. In order to study the viability of the approach, we have conducted an empirical experimentation which shows that the sand pile is able to find near-optimal schedules by reacting differently to different conditions of workloads and architectures.