Analysis of CCME: Coevolutionary Dynamics, Automatic Problem Decomposition, and Regularization

In most real-world problems, we either know little about the problems or the problems are too complex to have a clear vision on how to decompose them by hand. Thus, it is usually desirable to have a method to automatically decompose a complex problem into a set of subproblems and assign one or more specialists to each subproblem. The cooperative coevolutionary mixture of experts (CCME) model was designed to automatically decompose problems by combining the global optimization power of cooperative coevolution with the divide-and-conquer ability of mixture of experts. This paper analyzes how CCME decomposes complex classification problems through a principal-component-analysis-based visualization tool. The visualization shows that CCME decomposes the problem by driving different experts toward different regions of the input space. The paper also investigates the effect of regularization, using learning by forgetting (LF), on CCME. LF significantly reduces the structural complexity of CCME while maintaining the classification accuracy.