Stochastic Integrated Actor–Critic for Deep Reinforcement Learning

We propose a deep stochastic actor–critic algorithm with an integrated network architecture and fewer parameters. We address stabilization of the learning procedure via an adaptive objective to the critic’s loss and a smaller learning rate for the shared parameters between the actor and the critic. Moreover, we propose a mixed on–off policy exploration strategy to speed up learning. Experiments illustrate that our algorithm reduces the sample complexity by 50%–93% compared with the state-of-the-art deep reinforcement learning (RL) algorithms twin delayed deep deterministic policy gradient (TD3), soft actor–critic (SAC), proximal policy optimization (PPO), advantage actor–critic (A2C), and interpolated policy gradient (IPG) over continuous control tasks LunarLander, BipedalWalker, BipedalWalkerHardCore, Ant, and Minitaur in the OpenAI Gym.