Network overlays for efficient control of large scale dynamic groups

Scalable data distribution in large-scale dynamic collaborative systems requires efficient, low overhead communication control. We propose efficient algorithms for clustering network nodes dynamically based on their communication interest. Several group communication architectures have been proposed to date without considering the constraints imposed by the communication infrastructure. Among these, distributed hash tables are scalable and resilient data structures used for data dissemination control. However DHT's are not optimized for high dynamics of network node interest and real-time end-to-end performance requirements. This paper proposes efficient control algorithms for large-scale collaborative systems optimized for scalability as well as end-to-end data dissemination. Network node communication interest is modeled as a multi-dimensional attribute space partitioned into interest cells mapped to multicast communication groups. The proposed control algorithms use proximity-based clustering of network nodes and hierarchical communication interest aggregation. We show that network overlay control algorithms achieve scalability and low overhead with a controlled degradation of end-to-end data path performance