A Parallel Homotopic Optimized Control Scheme of Uncertain Nonlinear Markov Jump Systems and Its Applications

This paper investigates the optimized control problem for nonlinear Markov jump systems with uncertain time-varying parameters. First, the Takagi-Sugeno fuzzy model is applied to describe the uncertain nonlinear Markov jump systems, where the identification of the uncertain time-varying parameters is reformulated as zero-sum game problems. Subsequently, the reinforcement learning-based scheme is employed to solve such zero-sum game problems and achieve optimization even when the system dynamics remain unknown. Afterward, a parallel homotopic scheme is first adopted for such systems. The assumption of an initial stabilizing gain in the policy iteration scheme may be removed in this scheme, naturally enhancing the generality of the developed method. Finally, the actual availability of the proposed parallel homotopic scheme is validated through a single-link robot arm and a series DC motor. Note to Practitioners—In actual production processes, nonlinear Markov jump systems are widely applied to describe modern industrial systems subject to parameter jumps. The study of the optimized control for such systems can provide considerable benefits. However, external constraints such as complex physical environments and cost limitations make it difficult to obtain stable control gains when the system dynamics are completely unknown. This challenge has hindered the application of many existing reinforcement learning-based optimized control algorithms that require initial stable control gains. Moreover, the algorithms that do not rely on initial stable control gains frequently encounter issues with long computation times, rendering them unsuitable for industrial environments demanding rapid responses. To overcome these challenges, a parallel homotopic scheme is proposed for the optimized control of nonlinear Markov jump systems. The proposed scheme effectively derives an optimized controller from ensuring relatively fast convergence under weakened conditions, where the system dynamics are unknown and no initial stable gain is available. Additionally, the scheme addresses the uncertain time-varying parameters commonly present in industrial systems by employing the robust control theory. The application of the proposed scheme to a single-link robot arm and a series DC motor demonstrates its effective control performance.