A Controllable Lifestyle Simulator for Use in Deep Reinforcement Learning Algorithms

Deep learning, especially deep reinforcement learning (DRL), has become one of the most useful tools for solving sequential decision-making problems. This is particularly relevant to recommender systems and drug administration in healthcare. This approach has the main drawback of requiring a lot of training, usually several millions of examples before it can learn an interesting policy. Therefore, in healthcare, it is crucial for training the neural network to use reliable and realistic simulations of users. These simulations can be measurements of the user’s vital signals throughout the day or the occurrence of lifestyle events such as sleeping, eating, etc. In this paper, we present a novel and highly generalizable simulation system based on state machines associated with probabilistic transitions to simulate the user’s lifestyle. Thanks to its random nature, the proposed approach can generate infinite yet realistic data from the simulation. It can also be controlled based on the simulation constraints, which is an important requirement for DRL algorithms. We demonstrate its usefulness and relevance in two simple cases of user behavior simulation using different types of input data. We also show its application for a DRL scenario involving behavior change interventions.