Modular neural network structure for skill growing and structure expansion

One desirable feature of an intelligent system is the capability of expansion of the system structure as well as the learned techniques. In this study, we investigate the development of such capability. The used neural network structure, sum-of-product neural network (SOPNN), is a modularized neural network. A SOPNN with an initial number of submodules is used to learn the primary skill. To develop more advanced techniques, one or more submodules are added to the network. Weights in the original structure are not altered. This preserves the previously learned skills. New and more complicated skills can be established using newly added submodules. It is expected that, using modular learning architecture, learning and skill growing can be more efficient. Although the purpose of this study is to resolve the problem of skill expansion upon completing design, it is noted that the scheme can also be applied to relax learning difficulty by decomposing the learning of a difficult skill into several stages. Attractive features of the new learning scheme include modular structure, system expansibility, and potential faster learning.