Classification capabilities of two-layer neural nets

The authors consider the classification capabilities of feedforward two-layer neural nets with a single hidden layer and having threshold units only; that is they consider the type of decision regions that two-layer nets are capable of forming in the input space. It had been asserted previously that such nets are capable of forming only convex decision regions or nonconvex but connected regions. The authors show that two-layer nets are capable of forming disconnected decision regions as well. In addition to giving examples of the phenomena, they explain why and how disconnected decision regions are formed. They also derive an expression for the number of cells in the input space that are to be grouped together to form the decision regions. This expression can be useful in deciding how many nodes to have in the first layer. The results have bearing on neural networks where the nonlinear elements are smooth (sigmoid) functions rather than threshold functions.<>