Designing neural networks based on structure decomposition

This paper discusses the theory and methods for structure decomposition of multioutput feedforward neural nets, i.e., whether a feedforward neural net can be decomposed and how to decompose it. We find that such decomposition does exist, and in a decomposed system each subnet will have fewer hidden units, so the system will generalize well. Based on above theory, we propose two methods for designing neural nets: structure decomposition based weight decay method (SDBWD) is a pruning algorithm and structure decomposition based dynamic nodes creation (SDBDNC) is a construction method. These two methods show better performance on coal mixing and multiconcept learning examples.