Efficient Distributed Joint Detection of Widespread Events in Large Networked Systems

The Internet has become a fundamental platform for virtually all social, economical and security activities in modern societies. Monitoring widespread events on the Internet has many important applications in social trend studies and distributed intrusion/attack detection. This paper studies the problem of distributed joint detection of widespread events observed by the collaborating network devices (called watchers). In order to work with a large number of watchers, the recent work requires a central coordinator to help detect the common events. The central coordinator however has the problems of single- point of failure, fairness and trust issue in coordinator placement, and communication bottleneck at the coordinator. This paper proposes a fully- distributed solution for joint detection of common events based on a peer-to-peer model without using a central coordinator. Our design adopts an iterative set-join process that follows the structure of a hypercube, which reduces the communication complexity from O(n m) to O(m log n), where m is the size of the largest event set at any device and n is the number of collaborating devices.