Optimizing Reinforcement Learning in Partially Observable Environments Using Compressed Suffix Memory Algorithm

Reinforcement learning in partially observable environments poses challenges due to limited and noisy observations. Traditional approaches like the Utile Suffix Memory (USM) algorithm suffer from inefficiencies and potential overfitting. In this paper, I propose the Compressed Suffix Memory (CSM) algorithm, designed to enhance state space generation and decision-making efficiency. CSM leverages heuristic information obtained from initial blind exploration of the environment to dynamically adjust tree depth and instance density thresholds. By incorporating Boltzmann sampling, CSM balances exploration and exploitation, thereby improving learning performance. Experimental results on benchmark mazes demonstrate that CSM outperforms USM in terms of learning speed and effectiveness, providing a promising advancement in reinforcement learning algorithms for complex, partially observable domains.