A Reward Scheme for Production Systems with Overlapping Conflict Sets

The reward-allocation problem for production systems in delayed-payoff situations is formalized in a conceptual model in which the environment of the system is a finite automaton. The environment state and the state of the system's local memory determine which productions are in the current conflict set. Productions are selected from the conflict set with probabilities proportional to their activations. Each selected production updates the local memory and furnishes the next input symbol to the environment. A reward scheme examines the payoff that is output by the environment and adjusts the activations in an attempt to increase average payoff per unit time. A reward scheme is safe if it is generally biased towards improvement. The notion of reward scheme safety is formalized, an asymptotically safe reward scheme is exhibited, and its safety is demonstrated. The demonstration is an analog of the proof of Fisher's fundamental theorem of natural selection.