Robust Finite-Horizon Optimal Control of Boolean Control Networks Under Disturbances via Q-Learning

This paper presents a Q-learning approach to solving the finite-horizon optimal control problem for Boolean control networks (BCNs) under disturbances. We first introduce a dynamic programming solution to establish the ground truth for the optimal control policy, which serves as a benchmark for validating our Q-learning algorithm. We then develop a robust Q-learning algorithm that learns the optimal control policy through interactions with the environment, even in the presence of disturbances. The proposed approach aims to minimize a general cost function over a finite time horizon, ensuring that the system transitions from an initial state to a desired terminal state while avoiding forbidden states and actions. We demonstrate the effectiveness of our approach through illustrative examples, including a case study of the Arabinose operon in Escherichia coli. Our results show that the Q-learning algorithm converges to the optimal solution, providing a practical and robust method for finite-horizon optimal control in BCNs.