Learning Intrinsically Motivated Transition Models for Autonomous Systems

To support long-term autonomy and rational decision making, robotic systems should be risk aware and actively maintain the fidelity of critical state information. This is particularly difficult in natural environments that are dynamic, noisy, and partially observable. To support autonomy, predictive probabilistic models of robot-object interaction can be used to guide the agent toward rewarding and controllable outcomes with high probability while avoiding undesired states and allowing the agent to be aware of the amount of risk associated with acting. In this paper, we propose an intrinsically motivated learning technique to model probabilistic transition functions in a manner that is task-independent and sample efficient. We model them as Aspect Transition Graphs (ATGs)—a state-dependent control roadmap that depends on transition probability functions grounded in the sensory and motor resources of the robot. Experimental data that changes the relative perspective of an actively-controlled RGB-D camera is used to train empirical models of the transition probability functions. Our experiments demonstrate that the transition function of the underlying Partially Observable Markov Decision Process (POMDP) can be acquired efficiently using intrinsically motivated structure learning approach.