An Extended False Data Injection Attack via Deep Reinforcement Learning: Attack Model and Countermeasures in Cyber-Physical Power Systems

False data injection attacks are commonly used to evade the bad data detector in cyber-physical power systems. This paper proposes an extended attack strategy and a deep reinforcement learning-based detection method. Traditional false data injection attacks aim to remain stealthy and avoid detection by conventional detection mechanisms. An extended load attack is introduced to increase the potential for damage. Directly adding an extended component directly to the measurement makes it easily detectable by bad data detector. Accordingly, the extended attack integrates the added component into the state variables to improve stealth. An optimization model for the extended components of the proposed attack is developed, along with a homologous matrix. Additionally, an online attack detection scheme is formulated as a partially observable Markov decision process problem. A deep reinforcement learning-based detection framework is proposed, featuring a compound reward designed to minimize false alarms and time delays. The proposed online detector extracts state features under varying operating conditions and generates a policy to determine whether the power grid is under attack. An extended Euclidean distance indicator and an adaptive weight matrix are also proposed in the dynamic state estimation to improve estimation or detection accuracy. Numerical experiments validate the effectiveness and robustness of the proposed deep reinforcement learning-based detection scheme in power systems. Note to Practitioners—This paper is motivated by the lack of research on modeling the destructive capabilities of cyber-attacks and the inaccuracy of anomaly detection methods for data integrity attacks. Existing approaches to modeling false data injection attacks primarily focus on the hidden features bypassing detection of bad data detection in cyber-physical power systems. This paper proposes a novel false data injection modeling approach, triggered by the spinning reserves of power grids. The proposed false data injection strategy mathematically characterizes extended stealth attack mechanisms and introduces an extended load attack to explore greater destructive potential. Considering the uncertain environments in perspective attackers and defenders, this study formulates the attack detection problem as a partially observable Markov decision process. This study then characterizes how such metrics of detectors can be efficiently computed, this can allow a defender to automatically learn or generate a detection threshold policy using the deep reinforcement learning, and distinguishes real false data injection attacks from system noises. To improve the detection performance, a novel indicator and an adaptive weight matrix are proposed to enhance learning efficiency of detectors. Numerical simulations suggest that the proposed detection scheme is feasible to both traditional and extended false data injection attacks, though it does not yet account for incomplete measurements. Future work will focus on designing attack detection strategies under conditions of incomplete knowledge of network topology and measurements.