On the Convergence and Ultimate Reliability of Iterated Neural Nets

The organization and reliability of large redundant nets of formal neurons are investigated. Each neuron has m inputs and n outputs and is capable of performing at any one time any member of any prescribed subset of the total possible set of 2
^2m·n
functions. Various methods for obtaining large nets of predictable structure and function through the medium of element reproduction are discussed. In the principal reproduction scheme considered, called ``complete iteration,'' each element ``grows'' a copy of the net of which it is a part and each new element does likewise, ad infinitum. The output or net function is studied as the number of ``iterations'' or ``reproductions'' increases. The concept of net convergence (or oscillation) which is equivalent to the condition achieved by the net when its set of net functions becomes stable (or oscillatory) is precisely defined. Theorems are proved which predict this behavior for various nets of general interest. A polynomial system is associated with each net which completely describes its probabilistic behavior. This system produces the Moore-Shannon h(p) function in the special case where the net stabilizes (converges) on only two functions. Conditions are given under which a net may be made arbitrarily reliable even though its constituent elements are arbitrarily unreliable.