Stochastic approximation by neural networks using the Radon and wavelet transforms

We consider a stochastic algorithm for evaluating the parameters of a single hidden-layer neural network. The constructions makes use of a random discretization of an appropriate integral representation of general smooth functions, based on the Radon and wavelet transforms. An upper bound on the performance of the algorithm in approximating functions in the Sobolev class is given. These results are strengthened with the derivation of lower bounds on the approximation error, which demonstrate the tightness of the bounds up to a logarithmic factor in the network size.