A method based on Kolmogorov complexity to improve the efficiency of strategy optimization with limited memory space

The pervasive application of digital computer in control and optimization techniques forces us to consider the constraint of limited memory space when dealing with large scale practical systems. As an example, we consider the famous Witsenhausen counterexample with the new constraint of limited memory space in this paper. The main difficulty is how to sample strategies that can be stored in the given memory space efficiently. The concept of Kolmogorov complexity measures the minimal memory space to store a strategy (i.e., simple strategies), but is incomputable. To overcome this difficulty, we propose a method based on ordered binary decision diagram to sample only simple strategies. Besides the high sampling efficiency which is demonstrated by numerical testing, the proposed sampling method can be easily combined with optimization algorithms and performance evaluation techniques. As an example, we show how to combine ordinal optimization, numerical integration, and the proposed sampling method to solve the Witsenhausen problem with the constraint of limited memory space. We hope this work can shed some insights to computer-based optimization problems with memory space constraint in a more general situation.