Target-Reaching Control with Obstacle Avoidance Based Future Perception SAC for Unmanned Bicycle

A novel approach to control the balance and obstacle avoidance of unmanned bicycles using Soft Actor-Critic (SAC) algorithm enhanced with future state predictions is proposed in this paper. For reinforcement learning control of unmanned bicycles, inherent delayed reward poses a significant challenge. To address this issue, we introduce a Transformer-based time series prediction model combined with an autoencoder for dimensionality reduction. This prediction module provides the SAC algorithm with foresight into future states, improving decision-making capabilities. The proposed Future Perception-SAC (FP-SAC) framework integrates the state prediction module with the SAC algorithm, enhancing both sample efficiency and policy stability. Simulation experiments demonstrate that the FP-SAC algorithm outperforms the traditional SAC in terms of reward accumulation and balance maintenance and it can significantly improve the control performance of complex dynamic systems.