Multistage-Signaling-Game-Based Camouflage Defense Strategy Using Reinforcement Learning to Mitigate the Anti-Honeypot Attack

In Internet of Things (IoT) environments, anti-honeypot attacks exploit device fingerprinting and protocol analysis to undermine traditional honeypot defenses. To this end, we introduce an innovative camouflage defense strategy that dynamically disguises real IoT devices as honeypots to mislead attackers and protect critical resources. We formalize this adversarial interaction using a multistage signaling game, thereby reversing the information asymmetry that typically favors attackers. Subsequently, we theoretically derive the perfect Bayesian Nash equilibrium (PBNE) under conditions of incomplete information, thereby confirming the existence of mixed-strategy equilibria. Practically, we develop a signaling game-based multiagent proximal policy optimization (SG-MAPPO) algorithm by integrating long short-term memory (LSTM) networks and Bayesian updates, overcoming challenges in long-term strategy optimization and parameter convergence in dynamic IoT contexts. Experiments demonstrate that SG-MAPPO-nps outperforms existing methods in terms of training efficiency and stability, effectively reducing attacker success rates and providing a proactive defense solution for IoT security.