S3A: State-Attention Inducing Adversarial Attacks on Closed-Box Proximal Policy Optimization Models

In the current era when the Internet of Things (IoT) is booming and various smart devices are closely interconnected to build a complex network system, the demand for efficient control and resource management strategies is extremely urgent. Proximal policy optimization (PPO), as a highly representative algorithm in deep reinforcement learning, has great potential in aspects, such as precise control of IoT devices, rational resource allocation, and intelligent interaction. However, in the practical applications of the IoT, PPO mostly exists as a closed-box model, which is vulnerable to adversarial attacks. Moreover, the continuous action space scenarios applicable to PPO further increase the difficulty of analysis and protection. Therefore, this article innovatively proposes a state-attention adversarial attack (S3A) for closed-box PPO models in continuous action space scenarios. This method is based on the model’s attention to states, has a low cost, integrates the parts of model extraction, action clustering, and decision-making interference of the model under specific states, and has a clear interference purpose. By constructing an IoT-related victim model in the MuJoCo environment provided by the Gym library and implementing adversarial attacks, experiments have found that under four repetitions in five simulation environments, S3A reduces the final reward of the victim model by an average of 48.5%, which strongly verifies the effectiveness and influence of this attack method.