Command and control of cyber weapons

With the development of autonomous malware and autonomous anti-malware, command and control of cyber weapons is becoming an important part of cyber defence. In the present paper we discuss the dangers of deploying and controlling intelligent cyber weapons in a unified setting, considering these weapons as intelligent agents. Command and control of intelligent agents causes new threats that are difficult to avoid due to the complexity of behaviour of agents. Situation awareness of agents must be improved and verified, or at least carefully tested with respect to safety of their behaviour. Several possible dangerous behaviours of cyber weapons are discussed in the talk: misunderstanding of a situation, misinterpretation of commands, and loss of contact and formation of unwanted coalitions. A specific threat is the formation of unwanted coalitions by proactive weapons. This can happen if they get too much autonomy in decision making. A scenario of insubordination of agents is presented, considering a longer time perspective. General conclusions are the following: the more intelligent software becomes the more difficult it will be to control it; when designing and developing new cyber weapons, one has to guarantee the appropriate control over these weapons under any circumstances. It is practically impossible to use formal methods for verifying the safety of intelligent cyber weapons for their users. Setting strict constraints on the behaviour of cyber weapons and their careful testing are necessary.