By Topic

Neural Networks and Learning Systems, IEEE Transactions on

Issue 2 • Date Feb. 2012

Filter Results

Displaying Results 1 - 23 of 23
  • Table of contents

    Publication Year: 2012 , Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (117 KB)  
    Freely Available from IEEE
  • IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS publication information

    Publication Year: 2012 , Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (40 KB)  
    Freely Available from IEEE
  • A Unified Self-Stabilizing Neural Network Algorithm for Principal and Minor Components Extraction

    Publication Year: 2012 , Page(s): 185 - 198
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (814 KB) |  | HTML iconHTML  

    Recently, many unified learning algorithms have been developed for principal component analysis and minor component analysis. These unified algorithms can be used to extract principal components and, if altered simply by the sign, can also serve as a minor component extractor. This is of practical significance in the implementations of algorithms. This paper proposes a unified self-stabilizing neural network learning algorithm for principal and minor components extraction, and studies the stability of the proposed unified algorithm via the fixed-point analysis method. The proposed unified self-stabilizing algorithm for principal and minor components extraction is extended for tracking the principal subspace (PS) and minor subspace (MS). The averaging differential equation and the energy function associated with the unified algorithm for tracking PS and MS are given. It is shown that the averaging differential equation will globally asymptotically converge to an invariance set, and the corresponding energy function exhibit a unique global minimum attained if and only if its state matrices span the PS or MS of the autocorrelation matrix of a vector data stream. It is concluded that the proposed unified algorithm for tracking PS and MS can efficiently track an orthonormal basis of the PS or MS. Simulations are carried out to further illustrate the theoretical results achieved. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability and Dissipativity Analysis of Static Neural Networks With Time Delay

    Publication Year: 2012 , Page(s): 199 - 210
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (481 KB) |  | HTML iconHTML  

    This paper is concerned with the problems of stability and dissipativity analysis for static neural networks (NNs) with time delay. Some improved delay-dependent stability criteria are established for static NNs with time-varying or time-invariant delay using the delay partitioning technique. Based on these criteria, several delay-dependent sufficient conditions are given to guarantee the dissipativity of static NNs with time delay. All the given results in this paper are not only dependent upon the time delay but also upon the number of delay partitions. Some examples are given to illustrate the effectiveness and reduced conservatism of the proposed results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On-Line Node Fault Injection Training Algorithm for MLP Networks: Objective Function and Convergence Analysis

    Publication Year: 2012 , Page(s): 211 - 222
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (360 KB) |  | HTML iconHTML  

    Improving fault tolerance of a neural network has been studied for more than two decades. Various training algorithms have been proposed in sequel. The on-line node fault injection-based algorithm is one of these algorithms, in which hidden nodes randomly output zeros during training. While the idea is simple, theoretical analyses on this algorithm are far from complete. This paper presents its objective function and the convergence proof. We consider three cases for multilayer perceptrons (MLPs). They are: (1) MLPs with single linear output node; (2) MLPs with multiple linear output nodes; and (3) MLPs with single sigmoid output node. For the convergence proof, we show that the algorithm converges with probability one. For the objective function, we show that the corresponding objective functions of cases (1) and (2) are of the same form. They both consist of a mean square errors term, a regularizer term, and a weight decay term. For case (3), the objective function is slight different from that of cases (1) and (2). With the objective functions derived, we can compare the similarities and differences among various algorithms and various cases. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Tracking Control for Nonaffine Systems: A Self-Organizing Approximation Approach

    Publication Year: 2012 , Page(s): 223 - 235
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (642 KB) |  | HTML iconHTML  

    This paper considers tracking control for single-input, single-output nonaffine dynamic systems. A performance-dependent self-organizing approximation-based approach is proposed. The designer specifies a positive tracking error criterion. The self-organizing approximation-based controller then monitors the tracking performance and adds basis elements only as needed to achieve the tracking specification. Even though the system is not affine, the approach is defined such that the approximated function is independent of the control variable u. Stability is proved and the self-organization is derived in a Lyapunov-based methodology. To illustrate certain novel aspects of the proposed controller, a numerical example is included. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Geometric Algorithms to Large Margin Classifier Based on Affine Hulls

    Publication Year: 2012 , Page(s): 236 - 246
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (502 KB) |  | HTML iconHTML  

    The geometric framework for binary data classification problems provides an intuitive foundation for the comprehension and application of geometric optimization algorithms, leading to practical solutions of real-world classification problems. In this paper, some theoretical results on the candidate extreme points of the notion of reduced affine hull (RAH) are introduced. These results allow the existing nearest point algorithms to be directly applied to solve both separable and inseparable classification problems based on RAHs successfully and efficiently. As the practical applications of the new theoretical results, the popular Gilbert-Schlesinger-Kozinec and Mitchell-Dem'yanov-Malozemov algorithms are presented to solve binary classification problems in the context of the RAH framework. The theoretical analysis and some experiments show that the proposed methods successfully achieve significant performance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Concurrent Subspace Width Optimization Method for RBF Neural Network Modeling

    Publication Year: 2012 , Page(s): 247 - 259
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1461 KB) |  | HTML iconHTML  

    Radial basis function neural networks (RBFNNs) are widely used in nonlinear function approximation. One of the challenges in RBFNN modeling is determining how to effectively optimize width parameters to improve approximation accuracy. To solve this problem, a width optimization method, concurrent subspace width optimization (CSWO), is proposed based on a decomposition and coordination strategy. This method decomposes the large-scale width optimization problem into several subspace optimization (SSO) problems, each of which has a single optimization variable and smaller training and validation data sets so as to greatly simplify optimization complexity. These SSOs can be solved concurrently, thus computational time can be effectively reduced. With top-level system coordination, the optimization of SSOs can converge to a consistent optimum, which is equivalent to the optimum of the original width optimization problem. The proposed method is tested with four mathematical examples and one practical engineering approximation problem. The results demonstrate the efficiency and robustness of CSWO in optimizing width parameters over the traditional width optimization methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive Multiregression in Reproducing Kernel Hilbert Spaces: The Multiaccess MIMO Channel Case

    Publication Year: 2012 , Page(s): 260 - 276
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (828 KB) |  | HTML iconHTML  

    This paper introduces a wide framework for online, i.e., time-adaptive, supervised multiregression tasks. The problem is formulated in a general infinite-dimensional reproducing kernel Hilbert space (RKHS). In this context, a fairly large number of nonlinear multiregression models fall as special cases, including the linear case. Any convex, continuous, and not necessarily differentiable function can be used as a loss function in order to quantify the disagreement between the output of the system and the desired response. The only requirement is the subgradient of the adopted loss function to be available in an analytic form. To this end, we demonstrate a way to calculate the subgradients of robust loss functions, suitable for the multiregression task. As it is by now well documented, when dealing with online schemes in RKHS, the memory keeps increasing with each iteration step. To attack this problem, a simple sparsification strategy is utilized, which leads to an algorithmic scheme of linear complexity with respect to the number of unknown parameters. A convergence analysis of the technique, based on arguments of convex analysis, is also provided. To demonstrate the capacity of the proposed method, the multiregressor is applied to the multiaccess multiple-input multiple-output channel equalization task for a setting with poor resources and nonavailable channel information. Numerical results verify the potential of the method, when its performance is compared with those of the state-of-the-art linear techniques, which, in contrast, use space-time coding, more antenna elements, as well as full channel information. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Modeling and Monitoring of Dynamic Processes

    Publication Year: 2012 , Page(s): 277 - 284
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (558 KB) |  | HTML iconHTML  

    In this paper, a new online monitoring approach is proposed for handling the dynamic problem in industrial batch processes. Compared to conventional methods, its contributions are as follows: (1) multimodes are separated correctly since the cross-mode correlations are considered and the common information is extracted; (2) the expensive computing load is avoided since only the specific information is calculated when a mode is monitored online; and (3) after that, two different subspaces are separated, and the common and specific subspace models are built and analyzed, respectively. The monitoring is carried out in the subspace. The corresponding confidence regions are constructed according to their respective models. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Synchronization Control for Nonlinear Stochastic Dynamical Networks: Pinning Impulsive Strategy

    Publication Year: 2012 , Page(s): 285 - 292
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (841 KB) |  | HTML iconHTML  

    In this paper, a new control strategy is proposed for the synchronization of stochastic dynamical networks with nonlinear coupling. Pinning state feedback controllers have been proved to be effective for synchronization control of state-coupled dynamical networks. We will show that pinning impulsive controllers are also effective for synchronization control of the above mentioned dynamical networks. Some generic mean square stability criteria are derived in terms of algebraic conditions, which guarantee that the whole state-coupled dynamical network can be forced to some desired trajectory by placing impulsive controllers on a small fraction of nodes. An effective method is given to select the nodes which should be controlled at each impulsive constants. The proportion of the controlled nodes guaranteeing the stability is explicitly obtained, and the synchronization region is also derived and clearly plotted. Numerical simulations are exploited to demonstrate the effectiveness of the pinning impulsive strategy proposed in this paper. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multistability of Neural Networks With Time-Varying Delays and Concave-Convex Characteristics

    Publication Year: 2012 , Page(s): 293 - 305
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (650 KB) |  | HTML iconHTML  

    In this paper, stability of multiple equilibria of neural networks with time-varying delays and concave-convex characteristics is formulated and studied. Some sufficient conditions are obtained to ensure that an n-neuron neural network with concave-convex characteristics can have a fixed point located in the appointed region. By means of an appropriate partition of the n-dimensional state space, when nonlinear activation functions of an n-neuron neural network are concave or convex in 2k+2m-1 intervals, this neural network can have (2k+2m-1)n equilibrium points. This result can be applied to the multiobjective optimal control and associative memory. In particular, several succinct criteria are given to ascertain multistability of cellular neural networks. These stability conditions are the improvement and extension of the existing stability results in the literature. A numerical example is given to illustrate the theoretical findings via computer simulations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Time-Frequency Approach to Underdetermined Blind Source Separation

    Publication Year: 2012 , Page(s): 306 - 316
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1222 KB) |  | HTML iconHTML  

    This paper presents a new time-frequency (TF) underdetermined blind source separation approach based on Wigner-Ville distribution (WVD) and Khatri-Rao product to separate N non-stationary sources from M(M <; N) mixtures. First, an improved method is proposed for estimating the mixing matrix, where the negative value of the auto WVD of the sources is fully considered. Then after extracting all the auto-term TF points, the auto WVD value of the sources at every auto-term TF point can be found out exactly with the proposed approach no matter how many active sources there are as long as N ≤ 2M-1. Further discussion about the extraction of auto-term TF points is made and finally the numerical simulation results are presented to show the superiority of the proposed algorithm by comparing it with the existing ones. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • ARPOP: An Appetitive Reward-Based Pseudo-Outer-Product Neural Fuzzy Inference System Inspired From the Operant Conditioning of Feeding Behavior in Aplysia

    Publication Year: 2012 , Page(s): 317 - 329
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (785 KB) |  | HTML iconHTML  

    Appetitive operant conditioning in Aplysia for feeding behavior via the electrical stimulation of the esophageal nerve contingently reinforces each spontaneous bite during the feeding process. This results in the acquisition of operant memory by the contingently reinforced animals. Analysis of the cellular and molecular mechanisms of the feeding motor circuitry revealed that activity-dependent neuronal modulation occurs at the interneurons that mediate feeding behaviors. This provides evidence that interneurons are possible loci of plasticity and constitute another mechanism for memory storage in addition to memory storage attributed to activity-dependent synaptic plasticity. In this paper, an associative ambiguity correction-based neuro-fuzzy network, called appetitive reward-based pseudo-outer-product-compositional rule of inference [ARPOP-CRI(S)], is trained based on an appetitive reward-based learning algorithm which is biologically inspired by the appetitive operant conditioning of the feeding behavior in Aplysia. A variant of the Hebbian learning rule called Hebbian concomitant learning is proposed as the building block in the neuro-fuzzy network learning algorithm. The proposed algorithm possesses the distinguishing features of the sequential learning algorithm. In addition, the proposed ARPOP-CRI(S) neuro-fuzzy system encodes fuzzy knowledge in the form of linguistic rules that satisfies the semantic criteria for low-level fuzzy model interpretability. ARPOP-CRI(S) is evaluated and compared against other modeling techniques using benchmark time-series datasets. Experimental results are encouraging and show that ARPOP-CRI(S) is a viable modeling technique for time-variant problem domains. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global Convergence of Online BP Training With Dynamic Learning Rate

    Publication Year: 2012 , Page(s): 330 - 341
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (614 KB) |  | HTML iconHTML  

    The online backpropagation (BP) training procedure has been extensively explored in scientific research and engineering applications. One of the main factors affecting the performance of the online BP training is the learning rate. This paper proposes a new dynamic learning rate which is based on the estimate of the minimum error. The global convergence theory of the online BP training procedure with the proposed learning rate is further studied. It is proved that: 1) the error sequence converges to the global minimum error; and 2) the weight sequence converges to a fixed point at which the error function attains its global minimum. The obtained global convergence theory underlies the successful applications of the online BP training procedure. Illustrative examples are provided to support the theoretical analysis. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive Computation Algorithm for RBF Neural Network

    Publication Year: 2012 , Page(s): 342 - 347
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (370 KB) |  | HTML iconHTML  

    A novel learning algorithm is proposed for nonlinear modelling and identification using radial basis function neural networks. The proposed method simplifies neural network training through the use of an adaptive computation algorithm (ACA). In addition, the convergence of the ACA is analyzed by the Lyapunov criterion. The proposed algorithm offers two important advantages. First, the model performance can be significantly improved through ACA, and the modelling error is uniformly ultimately bounded. Secondly, the proposed ACA can reduce computational cost and accelerate the training speed. The proposed method is then employed to model classical nonlinear system with limit cycle and to identify nonlinear dynamic system, exhibiting the effectiveness of the proposed algorithm. Computational complexity analysis and simulation results demonstrate its effectiveness. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asynchronous Event-Based Binocular Stereo Matching

    Publication Year: 2012 , Page(s): 347 - 353
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (738 KB) |  | HTML iconHTML  

    We present a novel event-based stereo matching algorithm that exploits the asynchronous visual events from a pair of silicon retinas. Unlike conventional frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events, in a manner similar to the output cells of the biological retina. Our algorithm uses the timing information carried by this representation in addressing the stereo-matching problem on moving objects. Using the high temporal resolution of the acquired data stream for the dynamic vision sensor, we show that matching on the timing of the visual events provides a new solution to the real-time computation of 3-D objects when combined with geometric constraints using the distance to the epipolar lines. The proposed algorithm is able to filter out incorrect matches and to accurately reconstruct the depth of moving objects despite the low spatial resolution of the sensor. This brief sets up the principles for further event-based vision processing and demonstrates the importance of dynamic information and spike timing in processing asynchronous streams of visual events. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Frames for Exact Inversion of the Rank Order Coder

    Publication Year: 2012 , Page(s): 353 - 359
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (727 KB) |  | HTML iconHTML  

    Our goal is to revisit rank order coding by proposing an original exact decoding procedure for it. Rank order coding was proposed by Thorpe . who stated that the order in which the retina cells are activated encodes for the visual stimulus. Based on this idea, the authors proposed in a rank order coder/decoder associated to a retinal model. Though, it appeared that the decoding procedure employed yields reconstruction errors that limit the model bit-cost/quality performances when used as an image codec. The attempts made in the literature to overcome this issue are time consuming and alter the coding procedure, or are lacking mathematical support and feasibility for standard size images. Here we solve this problem in an original fashion by using the frames theory, where a frame of a vector space designates an extension for the notion of basis. Our contribution is twofold. First, we prove that the analyzing filter bank considered is a frame, and then we define the corresponding dual frame that is necessary for the exact image reconstruction. Second, to deal with the problem of memory overhead, we design a recursive out-of-core blockwise algorithm for the computation of this dual frame. Our work provides a mathematical formalism for the retinal model under study and defines a simple and exact reverse transform for it with over than 265 dB of increase in the peak signal-to-noise ratio quality compared to . Furthermore, the framework presented here can be extended to several models of the visual cortical areas using redundant representations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Globally Convergent MC Algorithm With an Adaptive Learning Rate

    Publication Year: 2012 , Page(s): 359 - 365
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (438 KB) |  | HTML iconHTML  

    This brief deals with the problem of minor component analysis (MCA). Artificial neural networks can be exploited to achieve the task of MCA. Recent research works show that convergence of neural networks based MCA algorithms can be guaranteed if the learning rates are less than certain thresholds. However, the computation of these thresholds needs information about the eigenvalues of the autocorrelation matrix of data set, which is unavailable in online extraction of minor component from input data stream. In this correspondence, we introduce an adaptive learning rate into the OJAn MCA algorithm, such that its convergence condition does not depend on any unobtainable information, and can be easily satisfied in practical applications. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes

    Publication Year: 2012 , Page(s): 365 - 371
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (238 KB) |  | HTML iconHTML  

    Extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks which need not be neuron-like and perform well in both regression and classification applications. In this brief, we propose an ELM with adaptive growth of hidden nodes (AG-ELM), which provides a new approach for the automated design of networks. Different from other incremental ELMs (I-ELMs) whose existing hidden nodes are frozen when the new hidden nodes are added one by one, in AG-ELM the number of hidden nodes is determined in an adaptive way in the sense that the existing networks may be replaced by newly generated networks which have fewer hidden nodes and better generalization performance. We then prove that such an AG-ELM using Lebesgue p-integrable hidden activation functions can approximate any Lebesgue p-integrable function on a compact input set. Simulation results demonstrate and verify that this new approach can achieve a more compact network architecture than the I-ELM. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • IEEE Transactions on Neural Networks & Learning Systems-Special issue on learning in nonevoltionary & evolving environments

    Publication Year: 2012 , Page(s): 372
    Save to Project icon | Request Permissions | PDF file iconPDF (143 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Publication Year: 2012 , Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks information for authors

    Publication Year: 2012 , Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks and Learning Systems publishes technical articles that deal with the theory, design, and applications of neural networks and related learning systems.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Derong Liu
Institute of Automation
Chinese Academy of Sciences