Notification:
We are currently experiencing intermittent issues impacting performance. We apologize for the inconvenience.
By Topic

Neural Networks and Learning Systems, IEEE Transactions on

Issue 4 • Date April 2013

Filter Results

Displaying Results 1 - 21 of 21
  • Table of contents

    Publication Year: 2013 , Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (116 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks and Learning Systems publication information

    Publication Year: 2013 , Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (138 KB)  
    Freely Available from IEEE
  • Stability Analysis for Neural Networks With Time-Varying Delay Based on Quadratic Convex Combination

    Publication Year: 2013 , Page(s): 513 - 521
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (385 KB) |  | HTML iconHTML  

    In this paper, a novel method is developed for the stability problem of a class of neural networks with time-varying delay. New delay-dependent stability criteria in terms of linear matrix inequalities for recurrent neural networks with time-varying delay are derived by the newly proposed augmented simple Lyapunov-Krasovski functional. Different from previous results by using the first-order convex combination property, our derivation applies the idea of second-order convex combination and the property of quadratic convex function which is given in the form of a lemma without resorting to Jensen's inequality. A numerical example is provided to verify the effectiveness and superiority of the presented results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Factor Analysis of Auto-Associative Neural Networks With Application in Speaker Verification

    Publication Year: 2013 , Page(s): 522 - 528
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (356 KB) |  | HTML iconHTML  

    Auto-associative neural network (AANN) is a fully connected feed-forward neural network, trained to reconstruct its input at its output through a hidden compression layer, which has fewer numbers of nodes than the dimensionality of input. AANNs are used to model speakers in speaker verification, where a speaker-specific AANN model is obtained by adapting (or retraining) the universal background model (UBM) AANN, an AANN trained on multiple held out speakers, using corresponding speaker data. When the amount of speaker data is limited, this adaptation procedure may lead to overfitting as all the parameters of UBM-AANN are adapted. In this paper, we introduce and develop the factor analysis theory of AANNs to alleviate this problem. We hypothesize that only the weight matrix connecting the last nonlinear hidden layer and the output layer is speaker-specific, and further restrict it to a common low-dimensional subspace during adaptation. The subspace is learned using large amounts of development data, and is held fixed during adaptation. Thus, only the coordinates in a subspace, also known as i-vector, need to be estimated using speaker-specific data. The update equations are derived for learning both the common low-dimensional subspace and the i-vectors corresponding to speakers in the subspace. The resultant i-vector representation is used as a feature for the probabilistic linear discriminant analysis model. The proposed system shows promising results on the NIST-08 speaker recognition evaluation (SRE), and yields a 23% relative improvement in equal error rate over the previously proposed weighted least squares-based subspace AANNs system. The experiments on NIST-10 SRE confirm that these improvements are consistent and generalize across datasets. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Projection-Based Fast Learning Fully Complex-Valued Relaxation Neural Network

    Publication Year: 2013 , Page(s): 529 - 541
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (409 KB) |  | HTML iconHTML  

    This paper presents a fully complex-valued relaxation network (FCRN) with its projection-based learning algorithm. The FCRN is a single hidden layer network with a Gaussian-like sech activation function in the hidden layer and an exponential activation function in the output layer. For a given number of hidden neurons, the input weights are assigned randomly and the output weights are estimated by minimizing a nonlinear logarithmic function (called as an energy function) which explicitly contains both the magnitude and phase errors. A projection-based learning algorithm determines the optimal output weights corresponding to the minima of the energy function by converting the nonlinear programming problem into that of solving a set of simultaneous linear algebraic equations. The resultant FCRN approximates the desired output more accurately with a lower computational effort. The classification ability of FCRN is evaluated using a set of real-valued benchmark classification problems from the University of California, Irvine machine learning repository. Here, a circular transformation is used to transform the real-valued input features to the complex domain. Next, the FCRN is used to solve three practical problems: a quadrature amplitude modulation channel equalization, an adaptive beamforming, and a mammogram classification. Performance results from this paper clearly indicate the superior classification/approximation performance of the FCRN. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Granular Neural Networks: Concepts and Development Schemes

    Publication Year: 2013 , Page(s): 542 - 553
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (965 KB) |  | HTML iconHTML  

    In this paper, we introduce a concept of a granular neural network and develop its comprehensive design process. The proposed granular network is formed on the basis of a given (numeric) neural network whose structure is augmented by the formation of granular connections (being realized as intervals) spanned over the numeric ones. Owing to its simplicity of the underlying processing, the interval connections become an appealing alternative of information granules to clarify the main idea. We introduce a concept of information granularity and its quantification (viewed as a level of information granularity). Being treated as an essential design asset, the assumed level of information granularity is distributed (allocated) among the connections of the network in several different ways so that certain performance index becomes maximized. Due to the high dimensionality nature of some protocols of allocation of information granularity and the nature of the allocation process itself, single-objective versions of particle swarm optimization is considered a suitable optimization vehicle. As we are concerned with the granular output of the network, which has to be evaluated with regard to the numeric target of data, two criteria are considered; namely, coverage of numeric data and specificity of information granules (intervals). A series of numeric studies completed for synthetic data and data coming from the machine learning and StatLib repositories provide a useful insight into the effectiveness of the proposed algorithm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Noise-Shaping Gradient Descent-Based Online Adaptation Algorithms for Digital Calibration of Analog Circuits

    Publication Year: 2013 , Page(s): 554 - 565
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1361 KB) |  | HTML iconHTML  

    Analog circuits that are calibrated using digital-to-analog converters (DACs) use a digital signal processor-based algorithm for real-time adaptation and programming of system parameters. In this paper, we first show that this conventional framework for adaptation yields suboptimal calibration properties because of artifacts introduced by quantization noise. We then propose a novel online stochastic optimization algorithm called noise-shaping or ΣΔ gradient descent, which can shape the quantization noise out of the frequency regions spanning the parameter adaptation trajectories. As a result, the proposed algorithms demonstrate superior parameter search properties compared to floating-point gradient methods and better convergence properties than conventional quantized gradient-methods. In the second part of this paper, we apply the ΣΔ gradient descent algorithm to two examples of real-time digital calibration: 1) balancing and tracking of bias currents, and 2) frequency calibration of a band-pass Gm-C biquad filter biased in weak inversion. For each of these examples, the circuits have been prototyped in a 0.5- μm complementary metal-oxide-semiconductor process, and we demonstrate that the proposed algorithm is able to find the optimal solution even in the presence of spurious local minima, which are introduced by the nonlinear and non-monotonic response of calibration DACs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cluster Consensus in Discrete-Time Networks of Multiagents With Inter-Cluster Nonidentical Inputs

    Publication Year: 2013 , Page(s): 566 - 578
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (577 KB) |  | HTML iconHTML  

    In this paper, cluster consensus of multiagent systems is studied via inter-cluster nonidentical inputs. Here, we consider general graph topologies, which might be time-varying. The cluster consensus is defined by two aspects: intracluster synchronization, the state at which differences between each pair of agents in the same cluster converge to zero, and inter-cluster separation, the state at which agents in different clusters are separated. For intra-cluster synchronization, the concepts and theories of consensus, including the spanning trees, scramblingness, infinite stochastic matrix product, and Hajnal inequality, are extended. As a result, it is proved that if the graph has cluster spanning trees and all vertices self-linked, then the static linear system can realize intra-cluster synchronization. For the time-varying coupling cases, it is proved that if there exists T > 0 such that the union graph across any T-length time interval has cluster spanning trees and all graphs has all vertices self-linked, then the time-varying linear system can also realize intra-cluster synchronization. Under the assumption of common inter-cluster influence, a sort of inter-cluster nonidentical inputs are utilized to realize inter-cluster separation, such that each agent in the same cluster receives the same inputs and agents in different clusters have different inputs. In addition, the boundedness of the infinite sum of the inputs can guarantee the boundedness of the trajectory. As an application, we employ a modified non-Bayesian social learning model to illustrate the effectiveness of our results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Common Nature of Learning Between Back-Propagation and Hopfield-Type Neural Networks for Generalized Matrix Inversion With Simplified Models

    Publication Year: 2013 , Page(s): 579 - 592
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (859 KB) |  | HTML iconHTML  

    In this paper, two simple-structure neural networks based on the error back-propagation (BP) algorithm (i.e., BP-type neural networks, BPNNs) are proposed, developed, and investigated for online generalized matrix inversion. Specifically, the BPNN-L and BPNN-R models are proposed and investigated for the left and right generalized matrix inversion, respectively. In addition, for the same problem-solving task, two discrete-time Hopfield-type neural networks (HNNs) are developed and investigated in this paper. Similar to the classification of the presented BPNN-L and BPNN-R models, the presented HNN-L and HNN-R models correspond to the left and right generalized matrix inversion, respectively. Comparing the BPNN weight-updating formula with the HNN state-transition equation for the specific (i.e., left or right) generalized matrix inversion, we show that such two derived learning-expressions turn out to be the same (in mathematics), although the BP and Hopfield-type neural networks are evidently different from each other a great deal, in terms of network architecture, physical meaning, and training patterns. Numerical results with different illustrative examples further demonstrate the efficacy of the presented BPNNs and HNNs for online generalized matrix inversion and, more importantly, their common natures of learning. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Online Support Vector Machine Based on Convex Hull Vertices Selection

    Publication Year: 2013 , Page(s): 593 - 609
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (887 KB) |  | HTML iconHTML  

    The support vector machine (SVM) method, as a promising classification technique, has been widely used in various fields due to its high efficiency. However, SVM cannot effectively solve online classification problems since, when a new sample is misclassified, the classifier has to be retrained with all training samples plus the new sample, which is time consuming. According to the geometric characteristics of SVM, in this paper we propose an online SVM classifier called VS-OSVM, which is based on convex hull vertices selection within each class. The VS-OSVM algorithm has two steps: 1) the samples selection process, in which a small number of skeleton samples constituting an approximate convex hull in each class of the current training samples are selected and 2) the online updating process, in which the classifier is updated with newly arriving samples and the selected skeleton samples. From the theoretical point of view, the first d+1 (d is the dimension of the input samples) selected samples are proved to be vertices of the convex hull. This guarantees that the selected samples in our approach keep the greatest amount of information of the convex hull. From the application point of view, the new algorithm can update the classifier without reducing its classification performance. Experimental results on benchmark data sets have shown the validity and effectiveness of the VS-OSVM algorithm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimizing Spatial Filters by Minimizing Within-Class Dissimilarities in Electroencephalogram-Based Brain–Computer Interface

    Publication Year: 2013 , Page(s): 610 - 619
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1058 KB) |  | HTML iconHTML  

    A major challenge in electroencephalogram (EEG)-based brain-computer interfaces (BCIs) is the inherent nonstationarities in the EEG data. Variations of the signal properties from intra and inter sessions often lead to deteriorated BCI performances, as features extracted by methods such as common spatial patterns (CSP) are not invariant against the changes. To extract features that are robust and invariant, this paper proposes a novel spatial filtering algorithm called Kullback-Leibler (KL) CSP. The CSP algorithm only considers the discrimination between the means of the classes, but does not consider within-class scatters information. In contrast, the proposed KLCSP algorithm simultaneously maximizes the discrimination between the class means, and minimizes the within-class dissimilarities measured by a loss function based on the KL divergence. The performance of the proposed KLCSP algorithm is compared against two existing algorithms, CSP and stationary CSP (sCSP), using the publicly available BCI competition III dataset IVa and a large dataset from stroke patients performing neuro-rehabilitation. The results show that the proposed KLCSP algorithm significantly outperforms both the CSP and the sCSP algorithms, in terms of classification accuracy, by reducing within-class variations. This results in more compact and separable features. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Just-In-Time Classifiers for Recurrent Concepts

    Publication Year: 2013 , Page(s): 620 - 634
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2113 KB) |  | HTML iconHTML  

    Just-in-time (JIT) classifiers operate in evolving environments by classifying instances and reacting to concept drift. In stationary conditions, a JIT classifier improves its accuracy over time by exploiting additional supervised information coming from the field. In nonstationary conditions, however, the classifier reacts as soon as concept drift is detected; the current classification setup is discarded and a suitable one activated to keep the accuracy high. We present a novel generation of JIT classifiers able to deal with recurrent concept drift by means of a practical formalization of the concept representation and the definition of a set of operators working on such representations. The concept-drift detection activity, which is crucial in promptly reacting to changes exactly when needed, is advanced by considering change-detection tests monitoring both inputs and classes distributions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Least Square Regularized Regression in Sum Space

    Publication Year: 2013 , Page(s): 635 - 646
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (588 KB) |  | HTML iconHTML  

    This paper proposes a least square regularized regression algorithm in sum space of reproducing kernel Hilbert spaces (RKHSs) for nonflat function approximation, and obtains the solution of the algorithm by solving a system of linear equations. This algorithm can approximate the low- and high-frequency component of the target function with large and small scale kernels, respectively. The convergence and learning rate are analyzed. We measure the complexity of the sum space by its covering number and demonstrate that the covering number can be bounded by the product of the covering numbers of basic RKHSs. For sum space of RKHSs with Gaussian kernels, by choosing appropriate parameters, we tradeoff the sample error and regularization error, and obtain a polynomial learning rate, which is better than that in any single RKHS. The utility of this method is illustrated with two simulated data sets and five real-life databases. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Dynamic Sampling Approach to Training Neural Networks for Multiclass Imbalance Classification

    Publication Year: 2013 , Page(s): 647 - 660
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1036 KB) |  | HTML iconHTML  

    Class imbalance learning tackles supervised learning problems where some classes have significantly more examples than others. Most of the existing research focused only on binary-class cases. In this paper, we study multiclass imbalance problems and propose a dynamic sampling method (DyS) for multilayer perceptrons (MLP). In DyS, for each epoch of the training process, every example is fed to the current MLP and then the probability of it being selected for training the MLP is estimated. DyS dynamically selects informative data to train the MLP. In order to evaluate DyS and understand its strength and weakness, comprehensive experimental studies have been carried out. Results on 20 multiclass imbalanced data sets show that DyS can outperform the compared methods, including pre-sample methods, active learning methods, cost-sensitive methods, and boosting-type methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • New Parameter-Free Simplified Swarm Optimization for Artificial Neural Network Training and its Application in the Prediction of Time Series

    Publication Year: 2013 , Page(s): 661 - 665
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (164 KB) |  | HTML iconHTML  

    A new soft computing method called the parameter-free simplified swarm optimization (SSO)-based artificial neural network (ANN), or improved SSO for short, is proposed to adjust the weights in ANNs. The method is a modification of the SSO, and seeks to overcome some of the drawbacks of SSO. In the experiments, the iSSO is compared with five other famous soft computing methods, including the backpropagation algorithm, the genetic algorithm, the particle swarm optimization (PSO) algorithm, cooperative random learning PSO, and the SSO, and its performance is tested on five famous time-series benchmark data to adjust the weights of two ANN models (multilayer perceptron and single multiplicative neuron model). The experimental results demonstrate that iSSO is robust and more efficient than the other five algorithms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Distributed Consensus Tracking for Multiple Uncertain Nonlinear Strict-Feedback Systems Under a Directed Graph

    Publication Year: 2013 , Page(s): 666 - 672
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (469 KB) |  | HTML iconHTML  

    In this brief, we study the distributed consensus tracking control problem for multiple strict-feedback systems with unknown nonlinearities under a directed graph topology. It is assumed that the leader's output is time-varying and has been accessed by only a small fraction of followers in a group. The distributed dynamic surface design approach is proposed to design local consensus controllers in order to guarantee the consensus tracking between the followers and the leader. The function approximation technique using neural networks is employed to compensate unknown nonlinear terms induced from the controller design procedure. From the Lyapunov stability theorem, it is shown that the consensus errors are cooperatively semiglobally uniformly ultimately bounded and converge to an adjustable neighborhood of the origin. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Ensemble Pruning Using Spectral Coefficients

    Publication Year: 2013 , Page(s): 673 - 678
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (451 KB) |  | HTML iconHTML  

    Ensemble pruning aims to increase efficiency by reducing the number of base classifiers, without sacrificing and preferably enhancing performance. In this brief, a novel pruning paradigm is proposed. Two class supervised learning problems are pruned using a combination of first- and second-order Walsh coefficients. A comparison is made with other ordered aggregation pruning methods, using multilayer perceptron base classifiers. The Walsh pruning method is analyzed with the help of a model that shows the relationship between second-order coefficients and added classification error with respect to Bayes error. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • 2014 IEEE World Congress on Computational Intelligence

    Publication Year: 2013 , Page(s): 679
    Save to Project icon | Request Permissions | PDF file iconPDF (3429 KB)  
    Freely Available from IEEE
  • Open Access

    Publication Year: 2013 , Page(s): 680
    Save to Project icon | Request Permissions | PDF file iconPDF (1156 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Publication Year: 2013 , Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (125 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks information for authors

    Publication Year: 2013 , Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks and Learning Systems publishes technical articles that deal with the theory, design, and applications of neural networks and related learning systems.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Derong Liu
Institute of Automation
Chinese Academy of Sciences