GrayINT – Detection and Localization of Gray Failures via Hybrid In-band Network Telemetry

As the underlying infrastructure, data center networks (DCNs) provide an important platform for cloud computing and next-generation networks. However, due to the influence of many factors, such as network, hardware, and software, various failures may occur during the operation of the DCN, which brings great challenges to its reliability. Particularly, a more insidious type of network failure, known as gray failure, presents a significant threat to system stability. In response to this challenge, in this work, we propose GrayINT, a gray failure detection and localization system that leverages In-band Network Telemetry (INT) technology to monitor all paths within the data center network in real-time. To verify the feasibility of our design, we build a fat-tree data center network on a virtual network testbed. A large number of experimental results demonstrate that our system can monitor device and link status in real-time, achieving gray failure detection and localization.