Scheduled System Maintenance:
On May 6th, single article purchases and IEEE account management will be unavailable from 8:00 AM - 5:00 PM ET (12:00 - 21:00 UTC). We apologize for the inconvenience.
By Topic

Neural Networks, IEEE Transactions on

Issue 2 • Date March 1998

Filter Results

Displaying Results 1 - 12 of 12
  • Comments on "Accelerated learning algorithm for multilayer perceptrons: optimization layer by layer"

    Publication Year: 1998 , Page(s): 339 - 341
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (83 KB)  

    In the above paper by Ergezinger and Thomsen (ibid. vol.6 (1991)), a new method for training multilayer perceptron, called optimization layer by layer (OLL), was introduced. The present paper analyzes the performance of OLL. We show, from theoretical considerations, that the amount of work required with OLL-learning scales as the third power of the network size, compared with the square of the network size for commonly used conjugate gradient (CG) training algorithms. This theoretical estimate is confirmed through a practical example. Thus, although OLL is shown to function very well for small neural networks (less than about 500 weights per layer), it is slower than CG for large neural networks. Next, we show that OLL does not always improve on the accuracy that can be obtained with CG. It seems that the final accuracy that can be obtained depends strongly on the initial network weights. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Comments on "Theoretical analysis of evolutionary algorithms with an infinite population size in continuous space. I. Basic properties of selection and mutation" [with reply]

    Publication Year: 1998 , Page(s): 341 - 343
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (55 KB)  

    In this paper, Gao points out two crucial errors in the proof of the theorem on the convergence of genetic algorithms (GAs) in the above paper by Qi-Palmieri (ibid., vol.5 (1994)). He presents two counter examples and comments that the errors may cause misleading on the convergence nature of GAs. He demonstrates that the sequence of mutation probabilities that increases the probability mass of the average set does not necessarily increase the mean fitness. In reply, Qi-Palmieri points out that the Theorem 3 simply states that it is possible to find a sequence of mutation densities that still guarantees convergence. The proof of Theorem 3 may be incomplete, but the result is unquestionably solid. They conclude that Gao's counterexamples may not add much insight into the nature of the problem, and they would rather encourage constructive contributions to the difficult, still open problems of the evolutionary paradigm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author's reply

    Publication Year: 1998 , Page(s): 342 - 343
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (27 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On relative convergence properties of principal component analysis algorithms

    Publication Year: 1998 , Page(s): 319 - 329
    Cited by:  Papers (15)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB)  

    We investigate the convergence properties of two different stochastic approximation algorithms for principal component analysis, and analytically explain some commonly observed experimental results. In our analysis, we use the theory of stochastic approximation, and in particular the results of Fabian (1968), to explore the asymptotic mean square errors (AMSEs) of the algorithms. This study reveals the conditions under which the algorithms produce smaller AMSEs, and also the conditions under which one algorithm has a smaller AMSE than the other. Experimental study with multidimensional Gaussian data corroborate our analytical findings. We next explore the convergence rates of the two algorithms. Our experiments and an analytical explanation reveals the conditions under which the algorithms converge faster to the solution, and also the conditions under which one algorithm converges faster than the other View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural tree density estimation for novelty detection

    Publication Year: 1998 , Page(s): 330 - 338
    Cited by:  Papers (7)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB)  

    In this paper, a neural competitive learning tree is introduced as a computationally attractive scheme for adaptive density estimation and novelty detection. The learning rule yields equiprobable quantization of the input space and provides an adaptive focusing mechanism capable of tracking time-varying distributions. It is shown by simulation that the neural tree performs reasonably well while being much faster than any of the other competitive learning algorithms View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural-network design for small training sets of high dimension

    Publication Year: 1998 , Page(s): 266 - 280
    Cited by:  Papers (25)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    We introduce a statistically based methodology for the design of neural networks when the dimension d of the network input is comparable to the size n of the training set. If one proceeds straightforwardly, then one is committed to a network of complexity exceeding n. The result will be good performance on the training set but poor generalization performance when the network is presented with new data. To avoid this we need to select carefully the network architecture, including control over the input variables. Our approach to selecting a network architecture first selects a subset of input variables (features) using the nonparametric statistical process of difference-based variance estimation and then selects a simple network architecture using projection pursuit regression (PPR) ideas combined with the statistical idea of slicing inverse regression (SIR). The resulting network, which is then retrained without regard to the PPR/SIR determined parameters, is one of moderate complexity (number of parameters significantly less than n) whose performance on the training set can be expected to generalize well. The application of this methodology is illustrated in detail in the context of short-term forecasting of the demand for electric power from an electric utility View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm

    Publication Year: 1998 , Page(s): 308 - 318
    Cited by:  Papers (146)  |  Patents (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    Presents a detailed performance analysis of the minimal resource allocation network (M-RAN) learning algorithm, M-RAN is a sequential learning radial basis function neural network which combines the growth criterion of the resource allocating network (RAN) of Platt (1991) with a pruning strategy based on the relative contribution of each hidden unit to the overall network output. The resulting network leads toward a minimal topology for the RAN. The performance of this algorithm is compared with the multilayer feedforward networks (MFNs) trained with 1) a variant of the standard backpropagation algorithm, known as RPROP and 2) the dependence identification (DI) algorithm of Moody and Antsaklis (1996) on several benchmark problems in the function approximation and pattern classification areas. For all these problems, the M-RAN algorithm is shown to realize networks with far fewer hidden neurons with better or same approximation/classification accuracy. Further, the time taken for learning (training) is also considerably shorter as M-RAN does not require repeated presentation of the training data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal decision boundaries for M-QAM signal formats using neural classifiers

    Publication Year: 1998 , Page(s): 241 - 246
    Cited by:  Papers (5)  |  Patents (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (196 KB)  

    The application of neural classifiers for providing optimal decision boundaries of a warped and clustered M-QAM constellation affected by nonlinearity is analyzed in this paper. The classifier behavior, for the specific application, has been evaluated both by the carrier to noise ratio (CNR) degradation (ΔC/N) due to nonlinearity for a target error rate Pc=10-3, and more thoroughly by classical figures of merit of the pattern recognition theory such as classification confidence and generalization capability. The influence of the probability distribution of the training examples and the effects of activation functions' sharpness (namely the temperature of the net) have also been investigated. The results, obtained on a simulation basis, indicate optimal matching with respect to upper bounds evaluated with some minor simplifying hypothesis, even if the overall method's effectiveness can be adequate only for mild nonlinearity conditions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recognition of human head orientation based on artificial neural networks

    Publication Year: 1998 , Page(s): 257 - 265
    Cited by:  Papers (31)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (200 KB)  

    Humans easily recognize where another person is looking and often use this information for interspeaker coordination. We present a method based on three neural networks of the local linear map type which enables a computer to identify the head orientation of a user by learning from examples. One network is used for color segmentation, a second for localization of the face, and the third for the final recognition of the head orientation. The system works at a frame rate of one image per second on a common workstation, We analyze the accuracy achieved at different processing steps and discuss the usability of the approach in the context of a visual human-machine interface View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Morphological associative memories

    Publication Year: 1998 , Page(s): 281 - 293
    Cited by:  Papers (90)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (580 KB)  

    The theory of artificial neural networks has been successfully applied to a wide variety of pattern recognition problems. In this theory, the first step in computing the next state of a neuron or in performing the next layer neural network computation involves the linear operation of multiplying neural values by their synaptic strengths and adding the results. A nonlinear activation function usually follows the linear operation in order to provide for nonlinearity of the network and set the next state of the neuron. In this paper we introduce a novel class of artificial neural networks, called morphological neural networks, in which the operations of multiplication and addition are replaced by addition and maximum (or minimum), respectively. By taking the maximum (or minimum) of sums instead of the sum of products, morphological network computation is nonlinear before possible application of a nonlinear activation function. As a consequence, the properties of morphological neural networks are drastically different than those of traditional neural network models. The main emphasis of the research presented here is on morphological associative memories. We examine the computing and storage capabilities of morphological associative memories and discuss differences between morphological models and traditional semilinear models such as the Hopfield net View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Radial basis function networks and complexity regularization in function learning

    Publication Year: 1998 , Page(s): 247 - 256
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (544 KB)  

    We apply the method of complexity regularization to derive estimation bounds for nonlinear function estimation using a single hidden layer radial basis function network. Our approach differs from previous complexity regularization neural-network function learning schemes in that we operate with random covering numbers and l1 metric entropy, making it possible to consider much broader families of activation functions, namely functions of bounded variation. Some constraints previously imposed on the network parameters are also eliminated this way. The network is trained by means of complexity regularization involving empirical risk minimization. Bounds on the expected risk in terms of the sample size are obtained for a large class of loss functions. Rates of convergence to the optimal loss are also derived View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Runge-Kutta neural network for identification of dynamical systems in high accuracy

    Publication Year: 1998 , Page(s): 294 - 307
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (652 KB)  

    This paper proposes Runge-Kutta neural networks (RKNNs) for identification of unknown dynamical systems described by ordinary differential equations (i.e., ordinary differential equation or ODE systems) with high accuracy. These networks are constructed according to the Runge-Kutta approximation method. The main attraction of the RKNNs is that they precisely estimate the changing rates of system states (i.e., the right-hand side of the ODE x˙=f(x)) directly in their subnetworks based on the space-domain interpolation within one sampling interval such that they can do long-term prediction of system state trajectories. We show theoretically the superior generalization and long-term prediction capability of the RKNNs over the normal neural networks. Two types of learning algorithms are investigated for the RKNNs, gradient-and nonlinear recursive least-squares-based algorithms. Convergence analysis of the learning algorithms is done theoretically. Computer simulations demonstrate the proved properties of the RKNNs View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope