Improved Soft Actor-Critic: Reducing Bias and Estimation Error for Fast Learning

Soft actor critic (SAC) is an off policy deep reinfor-cment learning algorithm that learns by using the principle of maximum entropy regularization. It updates the policy stochas-tically. The actor-critic formulation is combined with off policy learning procedure in SAC which helped it to attain state-of-art performance. This paper suggests some methods to enhance the performance of SAC. Firstly, multi-step look ahead is suggested which deals with the estimation error of q - values followed by an approach of dual actor network and delayed updates of the target and policy networks which minimize the errors transferred during the training. The resulted ImprovedSAC is compared with the original SAC interms of maximum rewards obtained. The simulation results in the benchmark control tasks of MUJOCO environments shows the effectiveness of the suggested methods.