QPID: A Distributed Priority Queue with Item Locality

Distributed systems have evolved from highly regular systems to irregular systems that are not based on any specific architecture. Highly optimized distributed algorithms that are often variants of classic data structures have been developed for these systems. However, such approaches may not be effective for applications built on loosely coupled irregular distributed systems. In this paper, we look at distributed priority queues for such systems, where the locality of items cannot be altered and bottlenecks at any one node should be avoided. We present QPID, a new distributed priority queue algorithm for irregular distributed systems in which nodes communicate via message passing only. We introduce mechanisms to track and update the head of a logical distributed priority queue without moving items between nodes and avoiding bottlenecks at any single node. A novel lazy update scheme is employed for improving the efficiency of our algorithm. We present our analysis of QPID and argue its correctness.