By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date June 2008

Filter Results

Displaying Results 1 - 23 of 23
  • Table of contents

    Page(s): C1 - C4
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • Beyond Feedforward Models Trained by Backpropagation: A Practical Training Tool for a More Efficient Universal Approximator

    Page(s): 929 - 937
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (859 KB) |  | HTML iconHTML  

    Cellular simultaneous recurrent neural network (SRN) has been shown to be a function approximator more powerful than the multilayer perceptron (MLP). This means that the complexity of MLP would be prohibitively large for some problems while SRN could realize the desired mapping with acceptable computational constraints. The speed of training of complex recurrent networks is crucial to their successful application. This work improves the previous results by training the network with extended Kalman filter (EKF). We implemented a generic cellular SRN (CSRN) and applied it for solving two challenging problems: 2-D maze navigation and a subset of the connectedness problem. The speed of convergence has been improved by several orders of magnitude in comparison with the earlier results in the case of maze navigation, and superior generalization has been demonstrated in the case of connectedness. The implications of this improvements are discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global Convergence and Limit Cycle Behavior of Weights of Perceptron

    Page(s): 938 - 947
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (441 KB) |  | HTML iconHTML  

    In this paper, it is found that the weights of a perceptron are bounded for all initial weights if there exists a nonempty set of initial weights that the weights of the perceptron are bounded. Hence, the boundedness condition of the weights of the perceptron is independent of the initial weights. Also, a necessary and sufficient condition for the weights of the perceptron exhibiting a limit cycle behavior is derived. The range of the number of updates for the weights of the perceptron required to reach the limit cycle is estimated. Finally, it is suggested that the perceptron exhibiting the limit cycle behavior can be employed for solving a recognition problem when downsampled sets of bounded training feature vectors are linearly separable. Numerical computer simulation results show that the perceptron exhibiting the limit cycle behavior can achieve a better recognition performance compared to a multilayer perceptron. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Centroid Neural Network With a Divergence Measure for GPDF Data Clustering

    Page(s): 948 - 957
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (682 KB) |  | HTML iconHTML  

    An unsupervised competitive neural network for efficient clustering of Gaussian probability density function (GPDF) data of continuous density hidden Markov models (CDHMMs) is proposed in this paper. The proposed unsupervised competitive neural network, called the divergence-based centroid neural network (DCNN), employs the divergence measure as its distance measure and utilizes the statistical characteristics of observation densities in the HMM for speech recognition problems. While the conventional clustering algorithms used for the vector quantization (VQ) codebook design utilize only the mean values of the observation densities in the HMM, the proposed DCNN utilizes both the mean and the covariance values. When compared with other conventional unsupervised neural networks, the DCNN successfully allocates more code vectors to the regions where GPDF data are densely distributed while it allocates fewer code vectors to the regions where GPDF data are sparsely distributed. When applied to Korean monophone recognition problems as a tool to reduce the size of the codebook, the DCNN reduced the number of GPDFs used for code vectors by 65.3% while preserving recognition accuracy. Experimental results with a divergence-based k-means algorithm and a divergence-based self-organizing map algorithm are also presented in this paper for a performance comparison. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Implementation of Pipelined FastICA on FPGA for Real-Time Blind Source Separation

    Page(s): 958 - 970
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3497 KB) |  | HTML iconHTML  

    Fast independent component analysis (FastICA) algorithm separates the independent sources from their mixtures by measuring non-Gaussian. FastICA is a common offline method to identify artifact and interference from their mixtures such as electroencephalogram (EEG), magnetoencephalography (MEG), and electrocardiogram (ECG). Therefore, it is valuable to implement FastICA for real-time signal processing. In this paper, the FastICA algorithm is implemented in a field-programmable gate array (FPGA), with the ability of real-time sequential mixed signals processing by the proposed pipelined FastICA architecture. Moreover, in order to increase the numbers precision, the hardware floating-point (FP) arithmetic units had been carried out in the hardware FastICA. In addition, the proposed pipeline FastICA provides the high sampling rate (192 kHz) capability by hand coding the hardware FastICA in hardware description language (HDL). To verify the features of the proposed hardware FastICA, simulations are first performed, then real-time signal processing experimental results are presented using the fabricated platform. Experimental results demonstrate the effectiveness of the presented hardware FastICA as expected. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global Convergence of SMO Algorithm for Support Vector Regression

    Page(s): 971 - 982
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (604 KB) |  | HTML iconHTML  

    Global convergence of the sequential minimal optimization (SMO) algorithm for support vector regression (SVR) is studied in this paper. Given l training samples, SVR is formulated as a convex quadratic programming (QP) problem with l pairs of variables. We prove that if two pairs of variables violating the optimality condition are chosen for update in each step and subproblems are solved in a certain way, then the SMO algorithm always stops within a finite number of iterations after finding an optimal solution. Also, efficient implementation techniques for the SMO algorithm are presented and compared experimentally with other SMO algorithms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimized Approximation Algorithm in Neural Networks Without Overfitting

    Page(s): 983 - 995
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1778 KB) |  | HTML iconHTML  

    In this paper, an optimized approximation algorithm (OAA) is proposed to address the overfitting problem in function approximation using neural networks (NNs). The optimized approximation algorithm avoids overfitting by means of a novel and effective stopping criterion based on the estimation of the signal-to-noise-ratio figure (SNRF). Using SNRF, which checks the goodness-of-fit in the approximation, overfitting can be automatically detected from the training error only without use of a separate validation set. The algorithm has been applied to problems of optimizing the number of hidden neurons in a multilayer perceptron (MLP) and optimizing the number of learning epochs in MLP's backpropagation training using both synthetic and benchmark data sets. The OAA algorithm can also be utilized in the optimization of other parameters of NNs. In addition, it can be applied to the problem of function approximation using any kind of basis functions, or to the problem of learning model selection when overfitting needs to be considered. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Constrained Optimization Approach to Preserving Prior Knowledge During Incremental Training

    Page(s): 996 - 1009
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1007 KB) |  | HTML iconHTML  

    In this paper, a supervised neural network training technique based on constrained optimization is developed for preserving prior knowledge of an input-output mapping during repeated incremental training sessions. The prior knowledge, referred to as long-term memory (LTM), is expressed in the form of equality constraints obtained by means of an algebraic training technique. Incremental training, which may be used to learn new short-term memories (STMs) online, is then formulated as an error minimization problem subject to equality constraints. The solution of this problem is simplified by implementing an adjoined error gradient that circumvents direct substitution and exploits classical backpropagation. A target application is neural network function approximation in adaptive critic designs. For illustrative purposes, constrained training is implemented to update an adaptive critic flight controller, while preserving prior knowledge of an established performance baseline that consists of classical gain-scheduled controllers. It is shown both analytically and numerically that the LTM is accurately preserved while the controller is repeatedly trained over time to assimilate new STMs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive Gain Control for Spike-Based Map Communication in a Neuromorphic Vision System

    Page(s): 1010 - 1021
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1279 KB) |  | HTML iconHTML  

    To support large numbers of model neurons, neuromorphic vision systems are increasingly adopting a distributed architecture, where different arrays of neurons are located on different chips or processors. Spike-based protocols are used to communicate activity between processors. The spike activity in the arrays depends on the input statistics as well as internal parameters such as time constants and gains. In this paper, we investigate strategies for automatically adapting these parameters to maintain a constant firing rate in response to changes in the input statistics. We find that under the constraint of maintaining a fixed firing rate, a strategy based upon updating the gain alone performs as well as an optimal strategy where both the gain and the time constant are allowed to vary. We discuss how to choose the time constant and propose an adaptive gain control mechanism whose operation is robust to changes in the input statistics. Our experimental results on a mobile robotic platform validate the analysis and efficacy of the proposed strategy. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Local Convergence Analysis of FastICA and Related Algorithms

    Page(s): 1022 - 1032
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (507 KB) |  | HTML iconHTML  

    The FastICA algorithm is one of the most prominent methods to solve the problem of linear independent component analysis (ICA). Although there have been several attempts to prove local convergence properties of FastICA, rigorous analysis is still missing in the community. The major difficulty of analysis is because of the well-known sign-flipping phenomenon of FastICA, which causes the discontinuity of the corresponding FastICA map on the unit sphere. In this paper, by using the concept of principal fiber bundles, FastICA is proven to be locally quadratically convergent to a correct separation. Higher order local convergence properties of FastICA are also investigated in the framework of a scalar shift strategy. Moreover, as a parallelized version of FastICA, the so-called QR FastICA algorithm, which employs the QR decomposition (Gram-Schmidt orthonormalization process) instead of the polar decomposition, is shown to share similar local convergence properties with the original FastICA. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Representation of Nonlinear Random Transformations by Non-Gaussian Stochastic Neural Networks

    Page(s): 1033 - 1060
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (5845 KB) |  | HTML iconHTML  

    The learning capability of neural networks is equivalent to modeling physical events that occur in the real environment. Several early works have demonstrated that neural networks belonging to some classes are universal approximators of input-output deterministic functions. Recent works extend the ability of neural networks in approximating random functions using a class of networks named stochastic neural networks (SNN). In the language of system theory, the approximation of both deterministic and stochastic functions falls within the identification of nonlinear no-memory systems. However, all the results presented so far are restricted to the case of Gaussian stochastic processes (SPs) only, or to linear transformations that guarantee this property. This paper aims at investigating the ability of stochastic neural networks to approximate nonlinear input-output random transformations, thus widening the range of applicability of these networks to nonlinear systems with memory. In particular, this study shows that networks belonging to a class named non-Gaussian stochastic approximate identity neural networks (SAINNs) are capable of approximating the solutions of large classes of nonlinear random ordinary differential transformations. The effectiveness of this approach is demonstrated and discussed by some application examples. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Incremental Learning of Chunk Data for Online Pattern Classification Systems

    Page(s): 1061 - 1074
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (525 KB) |  | HTML iconHTML  

    This paper presents a pattern classification system in which feature extraction and classifier learning are simultaneously carried out not only online but also in one pass where training samples are presented only once. For this purpose, we have extended incremental principal component analysis (IPCA) and some classifier models were effectively combined with it. However, there was a drawback in this approach that training samples must be learned one by one due to the limitation of IPCA. To overcome this problem, we propose another extension of IPCA called chunk IPCA in which a chunk of training samples is processed at a time. In the experiments, we evaluate the classification performance for several large-scale data sets to discuss the scalability of chunk IPCA under one-pass incremental learning environments. The experimental results suggest that chunk IPCA can reduce the training time effectively as compared with IPCA unless the number of input attributes is too large. We study the influence of the size of initial training data and the size of given chunk data on classification accuracy and learning time. We also show that chunk IPCA can obtain major eigenvectors with fairly good approximation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Absolute Exponential Stability of Recurrent Neural Networks With Generalized Activation Function

    Page(s): 1075 - 1089
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (648 KB) |  | HTML iconHTML  

    In this paper, the recurrent neural networks (RNNs) with a generalized activation function class is proposed. In this proposed model, every component of the neuron's activation function belongs to a convex hull which is bounded by two odd symmetric piecewise linear functions that are convex or concave over the real space. All of the convex hulls are composed of generalized activation function classes. The novel activation function class is not only with a more flexible and more specific description of the activation functions than other function classes but it also generalizes some traditional activation function classes. The absolute exponential stability (AEST) of the RNN with a generalized activation function class is studied through three steps. The first step is to demonstrate the global exponential stability (GES) of the equilibrium point of original RNN with a generalized activation function being equivalent to that of RNN under all vertex functions of convex hull. The second step transforms the RNN under every vertex activation function into neural networks under an array of saturated linear activation functions. Because the GES of the equilibrium point of three systems are equivalent, the next stability analysis focuses on the GES of the equilibrium point of RNN system under an array of saturated linear activation functions. The last step is to study both the existence of equilibrium point and the GES of the RNN under saturated linear activation functions using the theory of M-matrix. In the end, a two-neuron RNN with a generalized activation function is constructed to show the effectiveness of our results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonnegative Matrix Factorization in Polynomial Feature Space

    Page(s): 1090 - 1100
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2206 KB) |  | HTML iconHTML  

    Plenty of methods have been proposed in order to discover latent variables (features) in data sets. Such approaches include the principal component analysis (PCA), independent component analysis (ICA), factor analysis (FA), etc., to mention only a few. A recently investigated approach to decompose a data set with a given dimensionality into a lower dimensional space is the so-called nonnegative matrix factorization (NMF). Its only requirement is that both decomposition factors are nonnegative. To approximate the original data, the minimization of the NMF objective function is performed in the Euclidean space, where the difference between the original data and the factors can be minimized by employing L 2-norm. In this paper, we propose a generalization of the NMF algorithm by translating the objective function into a Hilbert space (also called feature space) under nonnegativity constraints. With the help of kernel functions, we developed an approach that allows high-order dependencies between the basis images while keeping the nonnegativity constraints on both basis images and coefficients. Two practical applications, namely, facial expression and face recognition, show the potential of the proposed approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Automatic Relevance Determination for Identifying Thalamic Regions Implicated in Schizophrenia

    Page(s): 1101 - 1107
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (699 KB) |  | HTML iconHTML  

    There have been many theories about and computational models of the Schizophrenic disease state. Brain imaging techniques have suggested that abnormalities of the Thalamus may contribute to the pathophysiology of Schizophrenia. Several studies have found the Thalamus to be altered in Schizophrenia, and the Thalamus has connections with other brain structures implicated in the disorder. This paper describes an experiment examining thalamic levels of the metabolite N-acetylaspartate (NAA), taken from schizophrenics and controls using in vivo proton magnetic resonance spectroscopic imaging. Automatic relevance determination was performed on neural networks trained on this data, identifying NAA group differences in the pulvinar and mediodorsal nucleus, underscoring the importance of examining thalamic subregions in schizophrenia. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Evaluation of the Traffic Parameters in a Metropolitan Area by Fusing Visual Perceptions and CNN Processing of Webcam Images

    Page(s): 1108 - 1129
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (5159 KB) |  | HTML iconHTML  

    This paper proposes a traffic monitoring architecture based on a high-speed communication network whose nodes are equipped with fuzzy processors and cellular neural network (CNN) embedded systems. It implements a real-time mobility information system where visual human perceptions sent by people working on the territory and video-sequences of traffic taken from Webcams are jointly processed to evaluate the fundamental traffic parameters for every street of a metropolitan area. This paper presents the whole methodology for data collection and analysis and compares the accuracy and the processing time of the proposed soft computing techniques with other existing algorithms. Moreover, this paper discusses when and why it is recommended to fuse the visual perceptions of the traffic with the automated measurements taken from the Webcams to compute the maximum traveling time that is likely needed to reach any destination in the traffic network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An Adaptive Learning Approach for 3-D Surface Reconstruction From Point Clouds

    Page(s): 1130 - 1140
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2120 KB) |  | HTML iconHTML  

    In this paper, we propose a multiresolution approach for surface reconstruction from clouds of unorganized points representing an object surface in 3-D space. The proposed method uses a set of mesh operators and simple rules for selective mesh refinement, with a strategy based on Kohonen's self-organizing map (SOM). Basically, a self-adaptive scheme is used for iteratively moving vertices of an initial simple mesh in the direction of the set of points, ideally the object boundary. Successive refinement and motion of vertices are applied leading to a more detailed surface, in a multiresolution, iterative scheme. Reconstruction was experimented on with several point sets, including different shapes and sizes. Results show generated meshes very close to object final shapes. We include measures of performance and discuss robustness. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Corrections to “On Adaptive Learning Rate That Guarantees Convergence in Feedforward Networks” [Sep 06 1116-1125]

    Page(s): 1141
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (35 KB)  

    In the above titled paper (ibid., vol. 17, no. 5, pp. 1116-1125), there were a few errors. Corrections are presented here. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neurodynamics of Cognition and Consciousness (Perlovsky, L.I. and Kozma, R., Eds.; 2007) [Book review]

    Page(s): 1142
    Save to Project icon | Request Permissions | PDF file iconPDF (28 KB)  
    Freely Available from IEEE
  • World Congress on Computational Intelligence - WCCI 2008

    Page(s): 1143
    Save to Project icon | Request Permissions | PDF file iconPDF (643 KB)  
    Freely Available from IEEE
  • Scitopia.org [advertisement]

    Page(s): 1144
    Save to Project icon | Request Permissions | PDF file iconPDF (269 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope