On PAC learning of functions with smoothness properties usingfeedforward sigmoidal networks
Rao, N.S.V.; Protopopescu, V.A.
Proceedings of the IEEE
Volume 84, Issue 10, Oct 1996 Page(s):1562 - 1569
Digital Object Identifier 10.1109/5.537119
Summary:We consider the problem of learning functions based on finite
samples by using feedforward sigmoidal networks. The unknown function f
is chosen from a family that has either bounded modulus of smoothness
and/or bounded capacity. The sample is given by (X1,
f(X1)), (X2, f(X2)), ...(Xn,
f(Xn)). Where X1, X2, ...,
Xn, are independently and identically distributed according
to an unknown distribution PX. General results guarantee the
existence of a neural network, fw*, that best approximates f
in terms of expected error. However, since both f and PX are
unknown, computing fw* is impossible in general. We propose
to compute probability and approximately correct (PAC) approximations to
fw*, based on alternative estimators, namely: 1) the nearest
neighbor rule, 2) local averaging, and 3) Nadaraya-Watson estimators,
all computed using the Haar system. We show that given a sufficiently
large sample, each of these estimators guarantees a performance as close
as desired to that of fw*. The practical importance of this
result sterns from the fact that, unlike neural networks, the three
estimators above are linear-time computable in terms of the sample size
View citation and abstract |
Cart
