LSTM-Based Anomalous Behavior Detection in Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning (MARL) extends individual reinforcement learning to enable a team of agents to collaboratively determine the global optimal policy that maximizes the sum of their local accumulated rewards. It has been recently deployed in multiple application domains such as edge computing, wireless networks, and Cyber-Physical Systems. Nonetheless, the security of MARL and its potential exposure to cyberattacks have not yet been fully investigated. This paper examines one of the most serious vulnerabilities in MARL algorithms: the compromised agent. This newly-engineered adversarial vulnerability is exploited when a malicious user compromises an agent to directly control its actions, and subsequently pushes its cooperative agents to act off-policy. We present a novel stacked-LSTM ensemble approach to detect such an attack. The results show that our anomalous behavior detection system significantly outperforms five baselines from the literature.