Reinforcement Learning based Multistage Optimal PMU Placement Against Data Integrity Attacks in Smart Grid

Data integrity attacks can mislead the state estimation of the smart grid and severely threaten the smart grid security. Phasor measurement units (PMUs) provide accurate measurements to state estimation and can be used to defend against data integrity attacks. Although defending against data integrity attacks and achieving observability of the smart grid using fewer PMUs have attracted increasing attention, few works on multistage optimal PMU placement strategies have been reported. Since the sequential decision making of reinforcement learning can provide a flexible PMU placement strategy, this paper designs a multistage optimal PMU placement strategy using reinforcement learning approach. By utilizing a least-effort attack model to find the most vulnerable buses that can be compromised by manipulating the minimum number of measurements, fewest PMUs are optimally placed in stages according to the designed reinforcement learning algorithm to ensure the smart grid is completely observable. To deal with the issue that the state space and the action space of the reinforcement learning approach in power grid environment are overlarge, a simple linear function needs smaller data space is introduced to approximate the value function. The effectiveness of the proposed multistage PMU placement strategy is verified on several IEEE standard test systems.