Adversarial Inverse Learning of Defense Policies Conditioned on Human Factor Models

The rapid growth of wireless networks has led to a significant increase in cybersecurity threats. Learning defense policies from offline data and expert demonstrations can eliminate the need to interact with a network/simulator during training, producing intelligent agents for automated/assistive defense. However, existing works on imitation and inverse learning are oblivious to various human factors that often affect decision-making and workflow in network defense. Learning defense policies that are conditioned on human factors can significantly improve the effectiveness of learned policies/models in network defense, better reflecting the varied behavior of human operators. To this end, we propose an adversarial inverse reinforcement learning algorithm incorporating models of human factors to learn user-specific policies/models in cyber defense from demonstrations and trajectories. The proposed algorithm captures varied operator behaviors through the modeling of three human factors: fatigue, expertise, and risk tolerance. Evaluations using the public Cyber Autonomy Gym for Experimentation (CAGE) Challenge 2 environment demonstrate significant improvement over baselines that are oblivious to underlying human factors.