Privacy-Cost Management in Smart Meters: Classical vs Deep Q-Learning with Mutual Information

In the realm of smart grids, a privacy-cost management framework was recently proposed, utilizing a modified deep double Q-learning algorithm that incorporates mutual information (MI) as the privacy metric. Numerous studies have confirmed the superior performance of deep reinforcement learning (DRL) frameworks compared to classical RL (CRL) approaches in safeguarding electricity consumption data, albeit at the expense of increased complexity. However, the significance of the privacy measure in the transition from classical RL (utilizing a simpler model) to deep RL (employing a more complex model) remains largely unexplored. This research aims to address this gap by investigating the importance of the privacy measure in the context of smart meter data privacy. Specifically, the privacy-cost management model based on the Q-learning algorithm with MI as the privacy metric is studied in two scenarios (CRL versus DRL). Our findings demonstrate that the classical Q-learning approach performs comparably to the deep Q-learning model while employing a simpler model architecture. This finding challenges previous research, which used flatness as a measure of privacy and favored DRL-based frameworks over CRL approaches. Our study underscores the crucial role of the privacy measure in determining the optimal approach.