An Evolutionary Approach to Quantify Internal States Needed for the Woods Problem

The Woods Problem is a difficult problem for purely reactive systems to handle. The difficulties are related to the perceptual aliasing problem, and the use of internal memory has been suggested to solve the problem. In this paper a novel approach in evolutionary computation is introduced to quantify the amount of memory required for a given task. The approach has been applied to Woods Problems such as wood101, woods102, Sutton's gridworld and woodsl4.
Finite state machine controllers are used, as these permit easy measurement of the amount of memory in the controller. A concurrent evolutionary search for the minimal but optimal control structure in memory-based systems, using an evolutionary Pareto-optimal search mechanism, determines the best behavior fitness for each level of controller memory. This memory analysis demonstrates the effect of internal memory in evolved controllers for Woods Problems and is also used to investigate the relationship between the number of sensors available to an agent and the amount of memory necessary for effective behavior.