Hierarchical Adaptive Control Framework for Autonomous Bicycles: Integrating Residual Decisions and Dynamic Optimization

This paper introduces the Adaptive Residual Decision-Control Synthesis (ARDCS) framework, a hierarchical control architecture that synergizes a model-based Linear Quadratic Regulator (LQR) with an adaptive Dynamic Proximal Policy Optimization (DPPO) agent for autonomous bicycle control. ARDCS is designed to take control of the bicycle’s complex nonlinear dynamics and adapt to environmental uncertainties by leveraging the stability of traditional control with the flexibility of reinforcement learning. A key innovation is a momentum-enhanced dynamic entropy adjustment mechanism within DPPO, which optimizes the exploration-exploitation trade-off for more stable and efficient learning. Comprehensive experiments on balancing and multi-target navigation tasks demonstrate that ARDCS consistently and significantly outperforms both conventional control methods and pure reinforcement learning strategies across varying levels of difficulty. The framework achieves enhanced adaptability and robust stability without relying on auxiliary mechanical stabilizers, offering a potent and generalizable solution for the control of under-actuated systems. Note to Practitioners—This research addresses the practical challenge of controlling inherently unstable two-wheeled vehicles in dynamic environments, particularly relevant for developing low-cost autonomous delivery robots and personal mobility devices. Our control framework enables bicycles to maintain balance and navigate complex obstacles without requiring expensive stabilization mechanisms, significantly reducing hardware complexity and energy consumption. The key innovation lies in combining traditional control theory with machine learning: a basic controller ensures fundamental stability while an AI layer handles complex decision-making. This approach has been successfully tested in simulations mimicking real-world conditions like uneven terrain and sudden wind gusts. Engineers can implement this solution using standard sensors and actuators, with potential applications extending to warehouse robots, drone stabilization, and adaptive industrial manipulators. Current limitations include computational demands for real-time decision-making, which may require optimization for low-power embedded systems. Future work will focus on simplifying the algorithm for edge computing devices while maintaining its adaptive capabilities. This technology could revolutionize last-mile delivery services and enable new categories of agile service robots within 3-5 years.