Reinforcement Learning-Based Accurate Worm Detection for Smart Grids

reinforcement learning (RL)-based worm detection chooses the test threshold to evaluate the network traffic features, such as the spectral flatness measure (SFM), but the detection of the evasive worm that modifies the scan rate and the worm propagation speed to manipulate the network traffic features is inaccurate due to the estimation and quantization error in the test threshold. In this article, we propose an RL-based accurate worm detection for smart grids that enables the control center to optimize the test threshold based on the number of meters, the infection time series and the number of connections to new destination IP addresses, besides the traffic log size received from each data concentrator and the number of the previously infected meters. A constraint on the maximum missed detection rate required by the smart grids is exploited in the detection policy distribution to support the reliable data transmission. A deep RL version addresses the quantization error in terms of the test threshold in the SFM evaluation and the infection time series in the state formulation, and compress the state space of the traffic log size received from a large number of data concentrators. Based on a worm detection game, the performance bound is provided under the specified detection window size and the propagation speed of evasive worm. Simulation results for 3600 m show that the performance gain of the detection accuracy and latency against evasive worm over the benchmarks.