By Topic

Neural Networks and Learning Systems, IEEE Transactions on

Issue 1 • Date Jan. 2013

Filter Results

Displaying Results 1 - 23 of 23
  • Table of contents

    Publication Year: 2013 , Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (119 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks and Learning Systems publication information

    Publication Year: 2013 , Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (138 KB)  
    Freely Available from IEEE
  • Editorial A Successful Change From TNN to TNNLS and a Very Successful Year

    Publication Year: 2013 , Page(s): 1 - 7
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | PDF file iconPDF (700 KB)  
    Freely Available from IEEE
  • Mixture Subclass Discriminant Analysis Link to Restricted Gaussian Model and Other Generalizations

    Publication Year: 2013 , Page(s): 8 - 21
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (455 KB) |  | HTML iconHTML  

    In this paper, a theoretical link between mixture subclass discriminant analysis (MSDA) and a restricted Gaussian model is first presented. Then, two further discriminant analysis (DA) methods, i.e., fractional step MSDA (FSMSDA) and kernel MSDA (KMSDA) are proposed. Linking MSDA to an appropriate Gaussian model allows the derivation of a new DA method under the expectation maximization (EM) framework (EM-MSDA), which simultaneously derives the discriminant subspace and the maximum likelihood estimates. The two other proposed methods generalize MSDA in order to solve problems inherited from conventional DA. FSMSDA solves the subclass separation problem, that is, the situation in which the dimensionality of the discriminant subspace is strictly smaller than the rank of the inter-between-subclass scatter matrix. This is done by an appropriate weighting scheme and the utilization of an iterative algorithm for preserving useful discriminant directions. On the other hand, KMSDA uses the kernel trick to separate data with nonlinearly separable subclass structure. Extensive experimentation shows that the proposed methods outperform conventional MSDA and other linear discriminant analysis variants. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Density-Preserving Sampling: Robust and Efficient Alternative to Cross-Validation for Error Estimation

    Publication Year: 2013 , Page(s): 22 - 34
    Save to Project icon | Click to expandQuick Abstract | PDF file iconPDF (920 KB) |  | HTML iconHTML  

    Estimation of the generalization ability of a classification or regression model is an important issue, as it indicates the expected performance on previously unseen data and is also used for model selection. Currently used generalization error estimation procedures, such as cross-validation (CV) or bootstrap, are stochastic and, thus, require multiple repetitions in order to produce reliable results, which can be computationally expensive, if not prohibitive. The correntropy-inspired density-preserving sampling (DPS) procedure proposed in this paper eliminates the need for repeating the error estimation procedure by dividing the available data into subsets that are guaranteed to be representative of the input dataset. This allows the production of low-variance error estimates with an accuracy comparable to 10 times repeated CV at a fraction of the computations required by CV. This method can also be used for model ranking and selection. This paper derives the DPS procedure and investigates its usability and performance using a set of public benchmark datasets and standard classifiers. View full abstract»

    Open Access
  • Two-Stage Nonnegative Sparse Representation for Large-Scale Face Recognition

    Publication Year: 2013 , Page(s): 35 - 46
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (708 KB) |  | HTML iconHTML  

    This paper proposes a novel nonnegative sparse representation approach, called two-stage sparse representation (TSR), for robust face recognition on a large-scale database. Based on the divide and conquer strategy, TSR decomposes the procedure of robust face recognition into outlier detection stage and recognition stage. In the first stage, we propose a general multisubspace framework to learn a robust metric in which noise and outliers in image pixels are detected. Potential loss functions, including L1 , L2,1, and correntropy are studied. In the second stage, based on the learned metric and collaborative representation, we propose an efficient nonnegative sparse representation algorithm to find an approximation solution of sparse representation. According to the L1 ball theory in sparse representation, the approximated solution is unique and can be optimized efficiently. Then a filtering strategy is developed to avoid the computation of the sparse representation on the whole large-scale dataset. Moreover, theoretical analysis also gives the necessary condition for nonnegative least squares technique to find a sparse solution. Extensive experiments on several public databases have demonstrated that the proposed TSR approach, in general, achieves better classification accuracy than the state-of-the-art sparse representation methods. More importantly, a significant reduction of computational costs is reached in comparison with sparse representation classifier; this enables the TSR to be more suitable for robust face recognition on a large-scale dataset. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Projection-Pursuit-Based Method for Blind Separation of Nonnegative Sources

    Publication Year: 2013 , Page(s): 47 - 57
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (765 KB) |  | HTML iconHTML  

    This paper presents a projection pursuit (PP) based method for blind separation of nonnegative sources. First, the available observation matrix is mapped to construct a new mixing model, in which the inaccessible source matrix is normalized to be column-sum-to-1. Then, the PP method is proposed to solve this new model, where the mixing matrix is estimated column by column through tracing the projections to the mapped observations in specified directions, which leads to the recovery of the sources. The proposed method is much faster than Chan's method, which has similar assumptions to ours, due to the usage of optimal projection. It is also more advantageous in separating cross-correlated sources than the independence- and uncorrelation-based methods, as it does not employ any statistical information of the sources. Furthermore, the new method does not require the mixing matrix to be nonnegative. Simulation results demonstrate the superior performance of our method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Synchronization for Coupled Neural Networks With Interval Delay: A Novel Augmented Lyapunov–Krasovskii Functional Method

    Publication Year: 2013 , Page(s): 58 - 70
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (548 KB) |  | HTML iconHTML  

    This paper is concerned with the synchronization problems for an array of neural networks with hybrid coupling and interval time-varying delay. First, a novel augmented Lyapunov-Krasovskii functional (LKF) method is proposed to develop delay-dependent synchronization criteria for the networks, which makes use of more relaxed conditions by employing the new type of augmented matrices with Kronecker product operation. The proposed method can handle a multitude of Kronecker product operations in the LKF and alleviates the requirements of the positive definiteness of some conditional matrices which are usually considered in the existing methods for complex networks. This leads to a significant improvement in the performance of the synchronization criteria, i.e., less conservative synchronization results can be obtained. Meanwhile, the case of fast time-varying delay can also be handled by the proposed method. Furthermore, based on the derived criteria, a robust synchronization criterion is obtained for the system with uncertainties both in coefficient and coupling matrix terms. Since an expression based on linear matrix inequality is used, the proposed criteria can be easily checked in practice. Finally, numerical examples are provided to show the effectiveness of the proposed method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Observer-Based Adaptive Neural Network Control for Nonlinear Stochastic Systems With Time Delay

    Publication Year: 2013 , Page(s): 71 - 80
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (456 KB) |  | HTML iconHTML  

    This paper considers the problem of observer-based adaptive neural network (NN) control for a class of single-input single-output strict-feedback nonlinear stochastic systems with unknown time delays. Dynamic surface control is used to avoid the so-called explosion of complexity in the backstepping design process. Radial basis function NNs are directly utilized to approximate the unknown and desired control input signals instead of the unknown nonlinear functions. The proposed adaptive NN output feedback controller can guarantee all the signals in the closed-loop system to be mean square semi-globally uniformly ultimately bounded. Simulation results are provided to demonstrate the effectiveness of the proposed methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Qualitative Adaptive Reward Learning With Success Failure Maps: Applied to Humanoid Robot Walking

    Publication Year: 2013 , Page(s): 81 - 93
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3294 KB) |  | HTML iconHTML  

    In the human brain, rewards are encoded in a flexible and adaptive way after each novel stimulus. Neurons of the orbitofrontal cortex are the key reward structure of the brain. Neurobiological studies show that the anterior cingulate cortex of the brain is primarily responsible for avoiding repeated mistakes. According to vigilance threshold, which denotes the tolerance to risks, we can differentiate between a learning mechanism that takes risks and one that averts risks. The tolerance to risk plays an important role in such a learning mechanism. Results have shown the differences in learning capacity between risk-taking and risk-avert behaviors. These neurological properties provide promising inspirations for robot learning based on rewards. In this paper, we propose a learning mechanism that is able to learn from negative and positive feedback with reward coding adaptively. It is composed of two phases: evaluation and decision making. In the evaluation phase, we use a Kohonen self-organizing map technique to represent success and failure. Decision making is based on an early warning mechanism that enables avoiding repeating past mistakes. The behavior to risk is modulated in order to gain experiences for success and for failure. Success map is learned with adaptive reward that qualifies the learned task in order to optimize the efficiency. Our approach is presented with an implementation on the NAO humanoid robot, controlled by a bioinspired neural controller based on a central pattern generator. The learning system adapts the oscillation frequency and the motor neuron gain in pitch and roll in order to walk on flat and sloped terrain, and to switch between them. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Novel Z-Domain Precoding Method for Blind Separation of Spatially Correlated Signals

    Publication Year: 2013 , Page(s): 94 - 105
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (516 KB) |  | HTML iconHTML  

    In this paper, we address the problem of blind separation of spatially correlated signals, which is encountered in some emerging applications, e.g., distributed wireless sensor networks and wireless surveillance systems. We preprocess the source signals in transmitters prior to transmission. Specifically, the source signals are first filtered by a set of properly designed precoders and then the coded signals are transmitted. On the receiving side, the Z-domain features of the precoders are exploited to separate the coded signals, from which the source signals are recovered. Based on the proposed precoders, a closed-form algorithm is derived to estimate the coded signals and the source signals. Unlike traditional blind source separation approaches, the proposed method does not require the source signals to be uncorrelated, sparse, or nonnegative. Compared with the existing precoder-based approach, the new method uses precoders with much lower order, which reduces the delay in data transmission and is easier to implement in practice. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Local Coordinates Alignment With Global Preservation for Dimensionality Reduction

    Publication Year: 2013 , Page(s): 106 - 117
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (576 KB) |  | HTML iconHTML  

    Dimensionality reduction is vital in many fields, and alignment-based methods for nonlinear dimensionality reduction have become popular recently because they can map the high-dimensional data into a low-dimensional subspace with the property of local isometry. However, the relationships between patches in original high-dimensional space cannot be ensured to be fully preserved during the alignment process. In this paper, we propose a novel method for nonlinear dimensionality reduction called local coordinates alignment with global preservation. We first introduce a reasonable definition of topology-preserving landmarks (TPLs), which not only contribute to preserving the global structure of datasets and constructing a collection of overlapping linear patches, but they also ensure that the right landmark is allocated to the new test point. Then, an existing method for dimensionality reduction that has good performance in preserving the global structure is used to derive the low-dimensional coordinates of TPLs. Local coordinates of each patch are derived using tangent space of the manifold at the corresponding landmark, and then these local coordinates are aligned into a global coordinate space with the set of landmarks in low-dimensional space as reference points. The proposed alignment method, called landmarks-based alignment, can produce a closed-form solution without any constraints, while most previous alignment-based methods impose the unit covariance constraint, which will result in the deficiency of global metrics and undesired rescaling of the manifold. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed algorithm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Hopf Bifurcation of an (n+1) -Neuron Bidirectional Associative Memory Neural Network Model With Delays

    Publication Year: 2013 , Page(s): 118 - 132
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (857 KB) |  | HTML iconHTML  

    Recent studies on Hopf bifurcations of neural networks with delays are confined to simplified neural network models consisting of only two, three, four, five, or six neurons. It is well known that neural networks are complex and large-scale nonlinear dynamical systems, so the dynamics of the delayed neural networks are very rich and complicated. Although discussing the dynamics of networks with a few neurons may help us to understand large-scale networks, there are inevitably some complicated problems that may be overlooked if simplified networks are carried over to large-scale networks. In this paper, a general delayed bidirectional associative memory neural network model with n+1 neurons is considered. By analyzing the associated characteristic equation, the local stability of the trivial steady state is examined, and then the existence of the Hopf bifurcation at the trivial steady state is established. By applying the normal form theory and the center manifold reduction, explicit formulae are derived to determine the direction and stability of the bifurcating periodic solution. Furthermore, the paper highlights situations where the Hopf bifurcations are particularly critical, in the sense that the amplitude and the period of oscillations are very sensitive to errors due to tolerances in the implementation of neuron interconnections. It is shown that the sensitivity is crucially dependent on the delay and also significantly influenced by the feature of the number of neurons. Numerical simulations are carried out to illustrate the main results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Prime Discriminant Simplicial Complex

    Publication Year: 2013 , Page(s): 133 - 144
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2334 KB) |  | HTML iconHTML  

    The structure representation of data distribution plays an important role in understanding the underlying mechanism of generating data. In this paper, we propose the prime discriminant simplicial complex (PDSC) by utilizing persistent homology to capture such structures. Assuming that each class is represented with a prime simplicial complex, we classify unlabeled samples based on the nearest projection distances from the samples to the simplicial complexes. We also extend the extrapolation ability of these complexes with a projection constraint term. Experiments in simulated and practical datasets indicate that, compared with several published algorithms, the proposed PDSC approaches achieve promising performance without losing structure representation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics

    Publication Year: 2013 , Page(s): 145 - 157
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1072 KB) |  | HTML iconHTML  

    To synthesize fixed-final-time control-constrained optimal controllers for discrete-time nonlinear control-affine systems, a single neural network (NN)-based controller called the Finite-horizon Single Network Adaptive Critic is developed in this paper. Inputs to the NN are the current system states and the time-to-go, and the network outputs are the costates that are used to compute optimal feedback control. Control constraints are handled through a nonquadratic cost function. Convergence proofs of: 1) the reinforcement learning-based training method to the optimal solution; 2) the training error; and 3) the network weights are provided. The resulting controller is shown to solve the associated time-varying Hamilton-Jacobi-Bellman equation and provide the fixed-final-time optimal solution. Performance of the new synthesis technique is demonstrated through different examples including an attitude control problem wherein a rigid spacecraft performs a finite-time attitude maneuver subject to control bounds. The new formulation has great potential for implementation since it consists of only one NN with single set of weights and it provides comprehensive feedback solutions online, though it is trained offline. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Feature Combiners With Gate-Generated Weights for Classification

    Publication Year: 2013 , Page(s): 158 - 163
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (222 KB) |  | HTML iconHTML  

    Using functional weights in a conventional linear combination architecture is a way of obtaining expressive power and represents an alternative to classical trainable and implicit nonlinear transformations. In this brief, we explore this way of constructing binary classifiers, taking advantage of the possibility of generating functional weights by means of a gate with fixed radial basis functions. This particular form of the gate permits training the machine directly with maximal margin algorithms. We call the resulting scheme “feature combiners with gate generated weights for classification.” Experimental results show that these architectures outperform support vector machines (SVMs) and Real AdaBoost ensembles in most considered benchmark examples. An increase in the computational design effort due to cross-validation demands is the price to be paid to obtain this advantage. Nevertheless, the operational effort is usually lower than that needed by SVMs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Competitive Learning With Pairwise Constraints

    Publication Year: 2013 , Page(s): 164 - 169
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (227 KB) |  | HTML iconHTML  

    Constrained clustering has been an active research topic since the last decade. Most studies focus on batch-mode algorithms. This brief introduces two algorithms for on-line constrained learning, named on-line linear constrained vector quantization error (O-LCVQE) and constrained rival penalized competitive learning (C-RPCL). The former is a variant of the LCVQE algorithm for on-line settings, whereas the latter is an adaptation of the (on-line) RPCL algorithm to deal with constrained clustering. The accuracy results-in terms of the normalized mutual information (NMI)-from experiments with nine datasets show that the partitions induced by O-LCVQE are competitive with those found by the (batch-mode) LCVQE. Compared with this formidable baseline algorithm, it is surprising that C-RPCL can provide better partitions (in terms of the NMI) for most of the datasets. Also, experiments on a large dataset show that on-line algorithms for constrained clustering can significantly reduce the computational time. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Infinite Hidden Conditional Random Fields for Human Behavior Analysis

    Publication Year: 2013 , Page(s): 170 - 177
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (345 KB) |  | HTML iconHTML  

    Hidden conditional random fields (HCRFs) are discriminative latent variable models that have been shown to successfully learn the hidden structure of a given classification problem (provided an appropriate validation of the number of hidden states). In this brief, we present the infinite HCRF (iHCRF), which is a nonparametric model based on hierarchical Dirichlet processes and is capable of automatically learning the optimal number of hidden states for a classification task. We show how we learn the model hyperparameters with an effective Markov-chain Monte Carlo sampling technique, and we explain the process that underlines our iHCRF model with the Restaurant Franchise Rating Agencies analogy. We show that the iHCRF is able to converge to a correct number of represented hidden states, and outperforms the best finite HCRFs-chosen via cross-validation-for the difficult tasks of recognizing instances of agreement, disagreement, and pain. Moreover, the iHCRF manages to achieve this performance in significantly less total training, validation, and testing time. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • 2014 IEEE World Congress on Computational Intelligence

    Publication Year: 2013 , Page(s): 178
    Save to Project icon | Request Permissions | PDF file iconPDF (3429 KB)  
    Freely Available from IEEE
  • Open Access

    Publication Year: 2013 , Page(s): 179
    Save to Project icon | Request Permissions | PDF file iconPDF (1156 KB)  
    Freely Available from IEEE
  • Proven powerful [advertisement]

    Publication Year: 2013 , Page(s): 180
    Save to Project icon | Request Permissions | PDF file iconPDF (188 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Publication Year: 2013 , Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks information for authors

    Publication Year: 2013 , Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks and Learning Systems publishes technical articles that deal with the theory, design, and applications of neural networks and related learning systems.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Derong Liu
Institute of Automation
Chinese Academy of Sciences