Reinforcement Learning Based Fast Worm Detection for Smart Grids

Smart grids suffer from worm attacks that inject worms to the vulnerable smart meters, make the infected meters propagate the infection to other vulnerable meters, and even damage the smart grids. In this paper, we present a deep reinforcement learning based worm detection scheme for smart grids without relying on the worm propagation model and network topology. This scheme measures the spectrum feature of the traffic in smart grids to detect the worms and applies two dueling neutral networks (NNs) to select the detection threshold based on the current state including the size of the traffic logs, the detection error rate in previous time slots and the estimated number of the infected meters. We apply an estimated NN and a target NN to select the detection threshold and calculate the expected long-term discount utility, respectively, to reduce the overestimation errors, and use the dueling architecture consisting of two fully connected layers to reduce the computational complexity. Simulation results show that the proposed scheme effectively reduces the detection error rate and worm detection latency.