For neural networks, function determines form

It is proved that, generically on nets, the I/O (input-output) behavior uniquely determines the internal form, up to simple symmetries. The sets where this conclusion does not hold are thin in the sense that they are included in sets defined by algebraic equalities. It is shown that, under very weak genericity assumptions, the following is true: assume given two nets, whose neurons all have the same nonlinear activation function sigma ; if the two sets have equal behaviors as 'black boxes', then necessarily they must have the same number of neurons and, except at most for sign reversals at each node, the same weights. The results obtained imply unique identifiability of parameters, under all possible I/O experiments. It is also possible to give a result showing that single experiments are (generically) sufficient for identification, in the analytic case. Some partial results can be obtained even if the precise nonlinearities are not known.<
>