A mathematical solution to a network construction problem

One of the major open issues in neural networks includes a network construction problem (NCP) to find a procedure, polynomial time if possible, that produces a minimal structure (minimum size, threshold and weight) of a multilayer threshold feed-forward network where its output must not exceed any given admissible distortion from a sample, The NCP includes a subproblem, a network training problem (NTP), where the size is prespecified. Approximate versions of the NCP/NTP have been solved with iterative algorithms for the network construction/training. This paper provides a mathematically rigorous solution to the NCP using rate distortion theory from information theory and linear algebra. This solution is used to develop a mathematical procedure that specifically constructs a minimal structure from the sample. The procedure attains the exact minimum though its computational time is, at worst, nonpolynomial. The paper also constructs a polynomial-time shortcut for approximate minimum sizes, which is a promising alternative to current algorithms.