Deep Reinforcement Learning Based Self-Evolving Moving Target Defense Approach Against Unknown Attacks

Deep reinforcement learning (DRL) based moving target defense (MTD) emerges as an outstanding method to enhance the security of highly hostile Internet of Things (IoT) environments. However, due to the gap between certain stationary learning environment and real-world, even a well-trained DRL model may not adapt to unknown attacks in the real-world network environments. Therefore, we present a DRL-based self-evolving MTD approach against unknown attacks. Firstly, we formulate the defense in a dynamic network environment as a Markov decision process (MDP), and utilize a DRL model based on actor-critic framework to obtain the optimal sequential defense strategies. Secondly, we deploy honeypots within the network environments to capture the traffic features of unknown attacks. These features are then specifically labeled to enable the DRL model to learn the characteristics of unknown attacks and identify them. Thirdly, we design an actor network based on ResNet architecture to perceive the network states and make defense decisions in response to these unknown attacks. Finally, through extensive experiments on a platform based on software defined networks, we show that our proposed approach can enhance the defense performance compared with state-of-the-art solutions when dealing with unknown attacks.