Towards Self-Defense of Non-Stationary Systems

One of the major trends in research on Intrusion Response Systems is to use a model of the system to be protected and/or a model of the attacker to predict the evolution of the system and of the strategy of the attacker. However, very often, modeled systems exhibit a non-stationary behavior due to changes in their configuration, in the software base and in the users behavior. If not properly captured by the system model, such a non-stationary behavior could lead to divergences between the expected and the actual behaviors, thus invalidating the model-based approach. In this paper, we introduce a model-free technique for self-defense of non-stationary systems based on Q-Learning. We experimentally show that the proposed approach is able to effectively capture the dynamics of the underlying system and quickly adapts to changes in the environment.