State space construction of reinforcement learning agents based upon anticipated sensory changes

We propose herein a new incremental state construction method which consists of Fritzke's growing neural gas algorithm and a class management mechanism of GNG units. The GNG algorithm condenses sensory inputs and learns which areas are frequently sensed. The CMM yields a new state based upon the anticipated behaviors of the agent, i.e., a couple of actions by an agent and the resultant change in sensory inputs. Computational simulations on the mountain-car task confirm the effectiveness of the proposed method.