Comparing the effectiveness of DQN and Q-Learning in terms of preserving the temperature within the desired temperature span.
Abstract:
Home energy management (HEM) systems optimize electricity demand of appliances according to the price-based demand response (DR) programs. Undoubtedly, customer satisfact...Show MoreMetadata
Abstract:
Home energy management (HEM) systems optimize electricity demand of appliances according to the price-based demand response (DR) programs. Undoubtedly, customer satisfaction is of such importance that if not taken into consideration, it prevents customers from participating in the DR. HEM systems suffer from high nonlinearity due to the variety of smart appliances and different criteria for customer satisfaction. In this paper, an advanced satisfaction-based HEM system using deep reinforcement learning is proposed to hourly schedule the controllable and time-shiftable appliances, including electric vehicle, air conditioner, and lighting system as controllable loads and washing machine, and dishwasher as time-shiftable loads. The proposed framework deploys a Deep Q-Network (DQN) method. Regarding customer dissatisfaction, this paper takes into consideration nonlinear precise functions. The Kano model for EV departure SoC, charging duration and lighting system satisfaction, desired temperature span for air conditioner, and the desirable operation period, waiting time, and consecutive mode of dishwasher and washing machine are taken into account. The proposed HEM system is applied to a smart home, and the results are compared with those of the Q-Learning algorithm. Numerical results prove the effectiveness of the proposed HEM system in reducing electricity cost and customer dissatisfaction, as well as the superiority of DQN over Q-Learning as well.
Comparing the effectiveness of DQN and Q-Learning in terms of preserving the temperature within the desired temperature span.
Published in: IEEE Access ( Volume: 10)

Department of Power and Control, School of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran
Ali Forootani received the B.Sc. degree in electrical power engineering from Shiraz University, Shiraz, Iran, in 2019, where he is currently pursuing the M.Sc. degree. His research interests include artificial intelligence, deep learning, reinforcement learning, energy management, and load forecasting.
Ali Forootani received the B.Sc. degree in electrical power engineering from Shiraz University, Shiraz, Iran, in 2019, where he is currently pursuing the M.Sc. degree. His research interests include artificial intelligence, deep learning, reinforcement learning, energy management, and load forecasting.View more

Department of Power and Control, School of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran
Mohammad Rastegar (Member, IEEE) received the B.Sc., M.Sc., and Ph.D. degrees in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2009, 2011, and 2015, respectively. He joined the School of Electrical and Computer Engineering, Shiraz University, in 2016. His current research interests include modeling home energy management systems, plug-in hybrid electric vehicle operation, and power syst...Show More
Mohammad Rastegar (Member, IEEE) received the B.Sc., M.Sc., and Ph.D. degrees in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2009, 2011, and 2015, respectively. He joined the School of Electrical and Computer Engineering, Shiraz University, in 2016. His current research interests include modeling home energy management systems, plug-in hybrid electric vehicle operation, and power syst...View more

Circular Economy Solutions Unit, Geologian Tutkimuskeskus (GTK), Espoo, Finland
Mohammad Jooshaki (Senior Member, IEEE) received the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2014, and the Ph.D. degree in power systems from Aalto University, Espoo, Finland, and the Sharif University of Technology, in 2020.
He is currently a Postdoctoral Researcher with the Circular Economy Solutions Unit, GTK, Espoo. His research interests include power system mo...Show More
Mohammad Jooshaki (Senior Member, IEEE) received the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2014, and the Ph.D. degree in power systems from Aalto University, Espoo, Finland, and the Sharif University of Technology, in 2020.
He is currently a Postdoctoral Researcher with the Circular Economy Solutions Unit, GTK, Espoo. His research interests include power system mo...View more

Department of Power and Control, School of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran
Ali Forootani received the B.Sc. degree in electrical power engineering from Shiraz University, Shiraz, Iran, in 2019, where he is currently pursuing the M.Sc. degree. His research interests include artificial intelligence, deep learning, reinforcement learning, energy management, and load forecasting.
Ali Forootani received the B.Sc. degree in electrical power engineering from Shiraz University, Shiraz, Iran, in 2019, where he is currently pursuing the M.Sc. degree. His research interests include artificial intelligence, deep learning, reinforcement learning, energy management, and load forecasting.View more

Department of Power and Control, School of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran
Mohammad Rastegar (Member, IEEE) received the B.Sc., M.Sc., and Ph.D. degrees in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2009, 2011, and 2015, respectively. He joined the School of Electrical and Computer Engineering, Shiraz University, in 2016. His current research interests include modeling home energy management systems, plug-in hybrid electric vehicle operation, and power system reliability and resiliency studies.
Mohammad Rastegar (Member, IEEE) received the B.Sc., M.Sc., and Ph.D. degrees in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2009, 2011, and 2015, respectively. He joined the School of Electrical and Computer Engineering, Shiraz University, in 2016. His current research interests include modeling home energy management systems, plug-in hybrid electric vehicle operation, and power system reliability and resiliency studies.View more

Circular Economy Solutions Unit, Geologian Tutkimuskeskus (GTK), Espoo, Finland
Mohammad Jooshaki (Senior Member, IEEE) received the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2014, and the Ph.D. degree in power systems from Aalto University, Espoo, Finland, and the Sharif University of Technology, in 2020.
He is currently a Postdoctoral Researcher with the Circular Economy Solutions Unit, GTK, Espoo. His research interests include power system modeling and optimization, distribution system reliability, performance-based regulations, and machine learning.
Mohammad Jooshaki (Senior Member, IEEE) received the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2014, and the Ph.D. degree in power systems from Aalto University, Espoo, Finland, and the Sharif University of Technology, in 2020.
He is currently a Postdoctoral Researcher with the Circular Economy Solutions Unit, GTK, Espoo. His research interests include power system modeling and optimization, distribution system reliability, performance-based regulations, and machine learning.View more