RL-Sketch: Scaling Reinforcement Learning for Adaptive and Automate Anomaly Detection in Network Data Streams

When network is undergoing problems, such as DDoS attack, component failures, etc., the detection of heavy flows (e.g. heavy hitters and heavy changers) is much more critical. However, it has been increasing challenging to ensure the accurate detection of heavy flows while dealing with massive network traffic volume, diversified traffic distribution and the stringent memory requirement. Although recent research efforts like LD-Sketch are scalable for diverse network traffic, they depend on excessive memory to maintain high accuracy, such that they fail to work well when the memory is limited. We propose RL-Sketch, a adaptive sketch using reinforcement learning in detecting heavy flows. It predicts potential heavy flows based on the statistics of network traffic, to achieve both high accuracy and scalability with minor memory. Trace-driven evaluation shows that RL-Sketch achieves higher accuracy than state-of-the-art sketch-based technologies with up to 17.79× accuracy gain, while maintaining high robustness in extreme conditions.