Loading web-font TeX/Math/Italic
Human-Level Control Through Directly Trained Deep Spiking Q-Networks | IEEE Journals & Magazine | IEEE Xplore

Human-Level Control Through Directly Trained Deep Spiking Q-Networks


Abstract:

As the third-generation neural networks, spiking neural networks (SNNs) have great potential on neuromorphic hardware because of their high energy efficiency. However, de...Show More

Abstract:

As the third-generation neural networks, spiking neural networks (SNNs) have great potential on neuromorphic hardware because of their high energy efficiency. However, deep spiking reinforcement learning (DSRL), that is, the reinforcement learning (RL) based on SNNs, is still in its preliminary stage due to the binary output and the nondifferentiable property of the spiking function. To address these issues, we propose a deep spiking Q -network (DSQN) in this article. Specifically, we propose a directly trained DSRL architecture based on the leaky integrate-and-fire (LIF) neurons and deep Q -network (DQN). Then, we adapt a direct spiking learning algorithm for the DSQN. We further demonstrate the advantages of using LIF neurons in DSQN theoretically. Comprehensive experiments have been conducted on 17 top-performing Atari games to compare our method with the state-of-the-art conversion method. The experimental results demonstrate the superiority of our method in terms of performance, stability, generalization and energy efficiency. To the best of our knowledge, our work is the first one to achieve state-of-the-art performance on multiple Atari games with the directly trained SNN.
Published in: IEEE Transactions on Cybernetics ( Volume: 53, Issue: 11, November 2023)
Page(s): 7187 - 7198
Date of Publication: 05 September 2022

ISSN Information:

PubMed ID: 36063509

Funding Agency:


References

References is not available for this document.