CARS: Dynamic Cyber-attack Reaction in SDN-based Networks with Q-learning

In this paper, we propose a dynamic cyber-attack reaction system based on Q-learning, namely CARS, to effectively defeat cyber-attacks in Software-Defined Networks (SDN). In particular, we first examine a cyber-attack reaction system that operates at the SDN control plane. Then, we propose a dynamic cyber-attack reaction solution to maximize the attack defense performance while minimizing the negative influence on benign traffic forwarding in the data plane. Next, we model the cyber-attack reaction system based on a Markov decision process (MDP) and formulate its optimization problem. Afterward, we develop a Q-learning based cyber-attack reaction control algorithm to solve the optimization problem, obtaining the optimal cyber-attack reaction policy. As our case study on denial-of-service (DoS) attacks, the obtained results verify that CARS can effectively prevent malicious packets from reaching the victim server in all DoS attacks, i.e., approximately 80% of abnormal packets are dropped. In addition, by implementing the optimal cyber-attack reaction policy, CARS can significantly reduce the ratio of QoS (Quality-of-Service) violated traffic flows compared to two existing solutions, i.e., GATE (by approx. 66%) and GTAC-IRS (by approx. 75%).