Control of Shared Production Buffers: A Reinforcement Learning Approach

We consider a buffer control problem arising in stochastic flow lines with shared production buffers. Buffer control relies on decision rules which determine transfers of items between buffers and machines at the release or completion times of parts on the different production stages. We devise a conceptual model of the problem for a basic scenario with one central buffer and explain how general system configurations and a tactical buffer allocation problem can be modeled within this framework. Assuming that the flow line can be represented as a Markovian production system, we provide a formulation as a continuous-time Markov decision problem admitting an optimal stationary policy. By applying a uniformization approach from literature, the Markov decision problem is discretized in time and thus amenable to standard algorithms. We propose a simple Q-learning implementation of reinforcement learning converging to an optimal stationary policy and validate the approach in a numerical experiment with a small toy problem.