By Topic

Neural Networks, IEEE Transactions on

Issue 4 • Date April 2011

Filter Results

Displaying Results 1 - 19 of 19
  • Table of contents

    Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (111 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE
  • Semisupervised Learning Using Bayesian Interpretation: Application to LS-SVM

    Page(s): 513 - 524
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (579 KB) |  | HTML iconHTML  

    Bayesian reasoning provides an ideal basis for representing and manipulating uncertain knowledge, with the result that many interesting algorithms in machine learning are based on Bayesian inference. In this paper, we use the Bayesian approach with one and two levels of inference to model the semisupervised learning problem and give its application to the successful kernel classifier support vector machine (SVM) and its variant least-squares SVM (LS-SVM). Taking advantage of Bayesian interpretation of LS-SVM, we develop a semisupervised learning algorithm for Bayesian LS-SVM using our approach based on two levels of inference. Experimental results on both artificial and real pattern recognition problems show the utility of our method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Model Construction of Boolean Network via Observed Data

    Page(s): 525 - 536
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (638 KB) |  | HTML iconHTML  

    In this paper, a set of data is assumed to be obtained from an experiment that satisfies a Boolean dynamic process. For instance, the dataset can be obtained from the diagnosis of describing the diffusion process of cancer cells. With the observed datasets, several methods to construct the dynamic models for such Boolean networks are proposed. Instead of building the logical dynamics of a Boolean network directly, its algebraic form is constructed first and then is converted back to the logical form. Firstly, a general construction technique is proposed. To reduce the size of required data, the model with the known network graph is considered. Motivated by this, the least in-degree model is constructed that can reduce the size of required data set tremendously. Next, the uniform network is investigated. The number of required data points for identification of such networks is independent of the size of the network. Finally, some principles are proposed for dealing with data with errors. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Iterative Gaussianization: From ICA to Random Rotations

    Page(s): 537 - 549
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1302 KB) |  | HTML iconHTML  

    Most signal processing problems involve the challenging task of multidimensional probability density function (PDF) estimation. In this paper, we propose a solution to this problem by using a family of rotation-based iterative Gaussianization (RBIG) transforms. The general framework consists of the sequential application of a univariate marginal Gaussianization transform followed by an orthonormal transform. The proposed procedure looks for differentiable transforms to a known PDF so that the unknown PDF can be estimated at any point of the original domain. In particular, we aim at a zero-mean unit-covariance Gaussian for convenience. RBIG is formally similar to classical iterative projection pursuit algorithms. However, we show that, unlike in PP methods, the particular class of rotations used has no special qualitative relevance in this context, since looking for interestingness is not a critical issue for PDF estimation. The key difference is that our approach focuses on the univariate part (marginal Gaussianization) of the problem rather than on the multivariate part (rotation). This difference implies that one may select the most convenient rotation suited to each practical application. The differentiability, invertibility, and convergence of RBIG are theoretically and experimentally analyzed. Relation to other methods, such as radial Gaussianization, one-class support vector domain description, and deep neural networks is also pointed out. The practical performance of RBIG is successfully illustrated in a number of multidimensional problems such as image synthesis, classification, denoising, and multi-information estimation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Online Blind Source Separation Using Incremental Nonnegative Matrix Factorization With Volume Constraint

    Page(s): 550 - 560
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1362 KB) |  | HTML iconHTML  

    Online blind source separation (BSS) is proposed to overcome the high computational cost problem, which limits the practical applications of traditional batch BSS algorithms. However, the existing online BSS methods are mainly used to separate independent or uncorrelated sources. Recently, nonnegative matrix factorization (NMF) shows great potential to separate the correlative sources, where some constraints are often imposed to overcome the non-uniqueness of the factorization. In this paper, an incremental NMF with volume constraint is derived and utilized for solving online BSS. The volume constraint to the mixing matrix enhances the identifiability of the sources, while the incremental learning mode reduces the computational cost. The proposed method takes advantage of the natural gradient based multiplication updating rule, and it performs especially well in the recovery of dependent sources. Simulations in BSS for dual-energy X-ray images, online encrypted speech signals, and high correlative face images show the validity of the proposed method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Convergence Dynamics of Stochastic Cohen–Grossberg Neural Networks With Unbounded Distributed Delays

    Page(s): 561 - 572
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (485 KB) |  | HTML iconHTML  

    This paper addresses the issue of the convergence dynamics of stochastic Cohen-Grossberg neural networks (SCGNNs) with white noise, whose state variables are described by stochastic nonlinear integro-differential equations. With the help of Lyapunov functional, semi-martingale theory, and inequality techniques, some novel sufficient conditions on pth moment exponential stability and almost sure exponential stability for SCGNN are given. Furthermore, as byproducts of our main results, some sufficient conditions for checking stability of deterministic CGNNs with unbounded distributed delays have been established. Especially, even when the spectral radius of the coefficient matrix is greater than 1, in some cases our theory is also effective. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier

    Page(s): 573 - 587
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (891 KB) |  | HTML iconHTML  

    Support vector machine (SVM), as one of the most popular classifiers, aims to find a hyperplane that can separate two classes of data with maximal margin. SVM classifiers are focused on achieving more separation between classes than exploiting the structures in the training data within classes. However, the structural information, as an implicit prior knowledge, has recently been found to be vital for designing a good classifier in different real-world problems. Accordingly, using as much prior structural information in data as possible to help improve the generalization ability of a classifier has yielded a class of effective structural large margin classifiers, such as the structured large margin machine (SLMM) and the Laplacian support vector machine (LapSVM). In this paper, we unify these classifiers into a common framework from the concept of “structural granularity” and the formulation for optimization problems. We exploit the quadratic programming (QP) and second-order cone programming (SOCP) methods, and derive a novel large margin classifier, we call the new classifier the structural regularized support vector machine (SRSVM). Unlike both SLMM at the cross of the cluster granularity and SOCP and LapSVM at the cross of the point granularity and QP, SRSVM is located at the cross of the cluster granularity and QP and thus follows the same optimization formulation as LapSVM to overcome large computational complexity and non-sparse solution in SLMM. In addition, it integrates the compactness within classes with the separability between classes simultaneously. Furthermore, it is possible to derive generalization bounds for these algorithms by using eigenvalue analysis of the kernel matrices. Experimental results demonstrate that SRSVM is often superior in classification and generalization performances to the state-of-the-art algorithms in the framework, both with the same and different structural granularities. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Convergence Study in Extended Kalman Filter-Based Training of Recurrent Neural Networks

    Page(s): 588 - 600
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (752 KB) |  | HTML iconHTML  

    Recurrent neural network (RNN) has emerged as a promising tool in modeling nonlinear dynamical systems, but the training convergence is still of concern. This paper aims to develop an effective extended Kalman filter-based RNN training approach with a controllable training convergence. The training convergence problem during extended Kalman filter-based RNN training has been proposed and studied by adapting two artificial training noise parameters: the covariance of measurement noise (R) and the covariance of process noise (Q) of Kalman filter. The R and Q adaption laws have been developed using the Lyapunov method and the maximum likelihood method, respectively. The effectiveness of the proposed adaption laws has been tested using a nonlinear dynamical benchmark system and further applied in cutting tool wear modeling. The results show that the R adaption law can effectively avoid the divergence problem and ensure the training convergence, whereas the Q adaption law helps improve the training convergence speed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Finite-Time Convergent Recurrent Neural Network With a Hard-Limiting Activation Function for Constrained Optimization With Piecewise-Linear Objective Functions

    Page(s): 601 - 613
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1617 KB) |  | HTML iconHTML  

    This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Pressure- and Work-Limited Neuroadaptive Control for Mechanical Ventilation of Critical Care Patients

    Page(s): 614 - 626
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (665 KB) |  | HTML iconHTML  

    In this paper, we develop a neuroadaptive control architecture to control lung volume and minute ventilation with input pressure constraints that also accounts for spontaneous breathing by the patient. Specifically, we develop a pressure - and work-limited neuroadaptive controller for mechanical ventilation based on a nonlinear multicompartmental lung model. The control framework does not rely on any averaged data and is designed to automatically adjust the input pressure to the patient's physiological characteristics capturing lung resistance and compliance modeling uncertainty. Moreover, the controller accounts for input pressure constraints as well as work of breathing constraints. Finally, the effect of spontaneous breathing is incorporated within the lung model and the control framework. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Design of a Neural Network Adaptive Controller via a Constrained Invariant Ellipsoids Technique

    Page(s): 627 - 638
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1320 KB) |  | HTML iconHTML  

    In safety critical applications, control architectures based on adaptive neural networks (NNs) must satisfy strict design specifications. This paper presents a practical approach for designing a mixed linear/adaptive model reference controller that recovers the performance of a reference model, and guarantees the boundedness of the tracking error within an a priori specified compact domain, in the presence of bounded uncertainties. The linear part of the controller results from the solution of an optimization problem where specifications are expressed as linear matrix inequality constraints. The linear controller is then augmented with a general adaptive NN that compensates for the uncertainties. The only requirement for the NN is that its output must be confined within pre-specified saturation limits. Toward this end a specific NN output confinement algorithm is proposed in this paper. The main advantages of the proposed approach are that requirements in terms of worst-case performance can be easily defined during the design phase, and that the design of the adaptation mechanism is largely independent from the synthesis of the linear controller. A numerical example is used to illustrate the design methodology. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiagent Reinforcement Learning: Spiking and Nonspiking Agents in the Iterated Prisoner's Dilemma

    Page(s): 639 - 653
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (647 KB) |  | HTML iconHTML  

    This paper investigates multiagent reinforcement learning (MARL) in a general-sum game where the payoffs' structure is such that the agents are required to exploit each other in a way that benefits all agents. The contradictory nature of these games makes their study in multiagent systems quite challenging. In particular, we investigate MARL with spiking and nonspiking agents in the Iterated Prisoner's Dilemma by exploring the conditions required to enhance its cooperative outcome. The spiking agents are neural networks with leaky integrate-and-fire neurons trained with two different learning algorithms: 1) reinforcement of stochastic synaptic transmission, or 2) reward-modulated spike-timing-dependent plasticity with eligibility trace. The nonspiking agents use a tabular representation and are trained with Q- and SARSA learning algorithms, with a novel reward transformation process also being applied to the Q-learning agents. According to the results, the cooperative outcome is enhanced by: 1) transformed internal reinforcement signals and a combination of a high learning rate and a low discount factor with an appropriate exploration schedule in the case of non-spiking agents, and 2) having longer eligibility trace time constant in the case of spiking agents. Moreover, it is shown that spiking and nonspiking agents have similar behavior and therefore they can equally well be used in a multiagent interaction setting. For training the spiking agents in the case where more than one output neuron competes for reinforcement, a novel and necessary modification that enhances competition is applied to the two learning algorithms utilized, in order to avoid a possible synaptic saturation. This is done by administering to the networks additional global reinforcement signals for every spike of the output neurons that were not “responsible” for the preceding decision. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Determination of Global Minima of Some Common Validation Functions in Support Vector Machine

    Page(s): 654 - 659
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (324 KB) |  | HTML iconHTML  

    Tuning of the regularization parameter C is a well-known process in the implementation of a support vector machine (SVM) classifier. Such a tuning process uses an appropriate validation function whose value, evaluated over a validation set, has to be optimized for the determination of the optimal C. Unfortunately, most common validation functions are not smooth functions of C. This brief presents a method for obtaining the global optimal solution of these non-smooth validation functions. The method is guaranteed to find the global optimum and relies on the regularization solution path of SVM over a range of C values. When the solution path is available, the computation needed is minimal. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Toward the Optimization of Normalized Graph Laplacian

    Page(s): 660 - 666
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (323 KB) |  | HTML iconHTML  

    Normalized graph Laplacian has been widely used in many practical machine learning algorithms, e.g., spectral clustering and semisupervised learning. However, all of them use the Euclidean distance to construct the graph Laplacian, which does not necessarily reflect the inherent distribution of the data. In this brief, we propose a method to directly optimize the normalized graph Laplacian by using pairwise constraints. The learned graph is consistent with equivalence and nonequivalence pairwise relationships, and thus it can better represent similarity between samples. Meanwhile, our approach, unlike metric learning, automatically determines the scale factor during the optimization. The learned normalized Laplacian matrix can be directly applied in spectral clustering and semisupervised learning algorithms. Comprehensive experiments demonstrate the effectiveness of the proposed approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global Robust Stability Criteria for Interval Delayed Full-Range Cellular Neural Networks

    Page(s): 666 - 671
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (330 KB) |  | HTML iconHTML  

    This brief considers a class of delayed full-range (FR) cellular neural networks (CNNs) with uncertain interconnections between neurons modeled by means of intervalized matrices. Using mathematical tools from the theory of differential inclusions, a fundamental result on global robust stability of standard (S) CNNs is extended to prove global robust exponential stability for the corresponding class (same interconnection weights and inputs) of FR-CNNs. The result is of theoretical interest since, in general, the equivalence between the dynamical behavior of FR-CNNs and S-CNNs is not guaranteed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Call for papers IEEE Transactions on Neural Networks Special Issue: Online Learning in Kernel Methods

    Page(s): 672
    Save to Project icon | Request Permissions | PDF file iconPDF (150 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks Information for authors

    Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope