By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date June 2009

Filter Results

Displaying Results 1 - 19 of 19
  • Table of contents

    Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (36 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • Probabilistic Classification Vector Machines

    Page(s): 901 - 914
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1245 KB) |  | HTML iconHTML  

    In this paper, a sparse learning algorithm, probabilistic classification vector machines (PCVMs), is proposed. We analyze relevance vector machines (RVMs) for classification problems and observe that adopting the same prior for different classes may lead to unstable solutions. In order to tackle this problem, a signed and truncated Gaussian prior is adopted over every weight in PCVMs, where the sign of prior is determined by the class label, i.e., +1 or -1. The truncated Gaussian prior not only restricts the sign of weights but also leads to a sparse estimation of weight vectors, and thus controls the complexity of the model. In PCVMs, the kernel parameters can be optimized simultaneously within the training algorithm. The performance of PCVMs is extensively evaluated on four synthetic data sets and 13 benchmark data sets using three performance metrics, error rate (ERR), area under the curve of receiver operating characteristic (AUC), and root mean squared error (RMSE). We compare PCVMs with soft-margin support vector machines (SVMSoft), hard-margin support vector machines (SVMHard), SVM with the kernel parameters optimized by PCVMs (SVMPCVM), relevance vector machines (RVMs), and some other baseline classifiers. Through five replications of twofold cross-validation F test, i.e., 5 times 2 cross-validation F test, over single data sets and Friedman test with the corresponding post-hoc test to compare these algorithms over multiple data sets, we notice that PCVMs outperform other algorithms, including SVMSoft, SVMHard, RVM, and SVMPCVM, on most of the data sets under the three metrics, especially under AUC. Our results also reveal that the performance of SVMPCVM is slightly better than SVMSoft, implying that the parameter optimization algorithm in PCVMs is better than cross validation in terms of performance and computational complexity. In this paper, we also discuss the superio- - rity of PCVMs' formulation using maximum a posteriori (MAP) analysis and margin analysis, which explain the empirical success of PCVMs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Delayed Projection Neural Network for Solving Linear Variational Inequalities

    Page(s): 915 - 925
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (728 KB) |  | HTML iconHTML  

    In this paper, a delayed projection neural network is proposed for solving a class of linear variational inequality problems. The theoretical analysis shows that the proposed neural network is globally exponentially stable under different conditions. By the proposed linear matrix inequality (LMI) method, the monotonicity assumption on the linear variational inequality is no longer necessary. By employing Lagrange multipliers, the proposed method can resolve the constrained quadratic programming problems. Finally, simulation examples are given to demonstrate the satisfactory performance of the proposed neural network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sparse Bayesian Modeling With Adaptive Kernel Learning

    Page(s): 926 - 937
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1127 KB) |  | HTML iconHTML  

    Sparse kernel methods are very efficient in solving regression and classification problems. The sparsity and performance of these methods depend on selecting an appropriate kernel function, which is typically achieved using a cross-validation procedure. In this paper, we propose an incremental method for supervised learning, which is similar to the relevance vector machine (RVM) but also learns the parameters of the kernels during model training. Specifically, we learn different parameter values for each kernel, resulting in a very flexible model. In order to avoid overfitting, we use a sparsity enforcing prior that controls the effective number of parameters of the model. We present experimental results on artificial data to demonstrate the advantages of the proposed method and we provide a comparison with the typical RVM on several commonly used regression and classification data sets. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust Dynamic Sliding-Mode Control Using Adaptive RENN for Magnetic Levitation System

    Page(s): 938 - 951
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3726 KB) |  | HTML iconHTML  

    In this paper, a robust dynamic sliding mode control system (RDSMC) using a recurrent Elman neural network (RENN) is proposed to control the position of a levitated object of a magnetic levitation system considering the uncertainties. First, a dynamic model of the magnetic levitation system is derived. Then, a proportional-integral-derivative (PID)-type sliding-mode control system (SMC) is adopted for tracking of the reference trajectories. Moreover, a new PID-type dynamic sliding-mode control system (DSMC) is proposed to reduce the chattering phenomenon. However, due to the hardware being limited and the uncertainty bound being unknown of the switching function for the DSMC, an RDSMC is proposed to improve the control performance and further increase the robustness of the magnetic levitation system. In the RDSMC, an RENN estimator is used to estimate an unknown nonlinear function of lumped uncertainty online and replace the switching function in the hitting control of the DSMC directly. The adaptive learning algorithms that trained the parameters of the RENN online are derived using Lyapunov stability theorem. Furthermore, a robust compensator is proposed to confront the uncertainties including approximation error, optimal parameter vectors, and higher order terms in Taylor series. Finally, some experimental results of tracking the various periodic trajectories demonstrate the validity of the proposed RDSMC for practical applications. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Permitted and Forbidden Sets in Discrete-Time Linear Threshold Recurrent Neural Networks

    Page(s): 952 - 963
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (685 KB) |  | HTML iconHTML  

    The concepts of permitted and forbidden sets enable a new perspective of the memory in neural networks. Such concepts exhibit interesting dynamics in recurrent neural networks. This paper studies the basic theories of permitted and forbidden sets of the linear threshold discrete-time recurrent neural networks. The linear threshold transfer function has been regarded as an adequate transfer function for recurrent neural networks. Networks with this transfer function form a class of hybrid analog and digital networks which are especially useful for perceptual computations. Networks in discrete time can directly provide algorithms for efficient implementation in digital hardware. The main contribution of this paper is to establish foundations of permitted and forbidden sets. Necessary and sufficient conditions for the linear threshold discrete-time recurrent neural networks are obtained for complete convergence, existence of permitted and forbidden sets, as well as conditionally multiattractivity, respectively. Simulation studies explore some possible interesting practical applications. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Associative Memory for Online Learning in Noisy Environments Using Self-Organizing Incremental Neural Network

    Page(s): 964 - 972
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1299 KB) |  | HTML iconHTML  

    Associative memory operating in a real environment must perform well in online incremental learning and be robust to noisy data because noisy associative patterns are presented sequentially in a real environment. We propose a novel associative memory that satisfies these requirements. Using the proposed method, new associative pairs that are presented sequentially can be learned accurately without forgetting previously learned patterns. The memory size of the proposed method increases adaptively with learning patterns. Therefore, it suffers neither redundancy nor insufficiency of memory size, even in an environment in which the maximum number of associative pairs to be presented is unknown before learning. Noisy inputs in real environments are classifiable into two types: noise-added original patterns and faultily presented random patterns. The proposed method deals with two types of noise. To our knowledge, no conventional associative memory addresses noise of both types. The proposed associative memory performs as a bidirectional one-to-many or many-to-one associative memory and deals not only with bipolar data, but also with real-valued data. Results demonstrate that the proposed method's features are important for application to an intelligent robot operating in a real environment. The originality of our work consists of two points: employing a growing self-organizing network for an associative memory, and discussing what features are necessary for an associative memory for an intelligent robot and proposing an associative memory that satisfies those requirements. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Neural Network Model to Minimize the Connected Dominating Set for Self-Configuration of Wireless Sensor Networks

    Page(s): 973 - 982
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1003 KB) |  | HTML iconHTML  

    A wireless ad hoc sensor network consists of a number of sensors spreading across a geographical area. The performance of the network suffers as the number of nodes grows, and a large sensor network quickly becomes difficult to manage. Thus, it is essential that the network be able to self-organize. Clustering is an efficient approach to simplify the network structure and to alleviate the scalability problem. One method to create clusters is to use weakly connected dominating sets (WCDSs). Finding the minimum WCDS in an arbitrary graph is an NP-complete problem. We propose a neural network model to find the minimum WCDS in a wireless sensor network. We present a directed convergence algorithm. The new algorithm outperforms the normal convergence algorithm both in efficiency and in the quality of solutions. Moreover, it is shown that the neural network is robust. We investigate the scalability of the neural network model by testing it on a range of sized graphs and on a range of transmission radii. Compared with Guha and Khuller's centralized algorithm, the proposed neural network with directed convergency achieves better results when the transmission radius is short, and equal performance when the transmission radius becomes larger. The parallel version of the neural network model takes time O(d) , where d is the maximal degree in the graph corresponding to the sensor network, while the centralized algorithm takes O(n 2). We also investigate the effect of the transmission radius on the size of WCDS. The results show that it is important to select a suitable transmission radius to make the network stable and to extend the lifespan of the network. The proposed model can be used on sink nodes in sensor networks, so that a sink node can inform the nodes to be a coordinator (clusterhead) in the WCDS obtained by the algorithm. Thus, the message overhead is O(M), where M is the size of the WCDS. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recurrent Neural Networks Training With Stable Bounding Ellipsoid Algorithm

    Page(s): 983 - 991
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (549 KB) |  | HTML iconHTML  

    Bounding ellipsoid (BE) algorithms offer an attractive alternative to traditional training algorithms for neural networks, for example, backpropagation and least squares methods. The benefits include high computational efficiency and fast convergence speed. In this paper, we propose an ellipsoid propagation algorithm to train the weights of recurrent neural networks for nonlinear systems identification. Both hidden layers and output layers can be updated. The stability of the BE algorithm is proven. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Spatio–Temporal Adaptation in the Unsupervised Development of Networked Visual Neurons

    Page(s): 992 - 1008
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3003 KB) |  | HTML iconHTML  

    There have been many computational models mimicking the visual cortex that are based on spatial adaptations of unsupervised neural networks. In this paper, we present a new model called neuronal cluster which includes spatial as well as temporal weights in its unified adaptation scheme. The ldquoin-placerdquo nature of the model is based on two biologically plausible learning rules, Hebbian rule and lateral inhibition. We present the mathematical demonstration that the temporal weights are derived from the delay in lateral inhibition. By training with the natural videos, this model can develop spatio-temporal features such as orientation selective cells, motion sensitive cells, and spatio-temporal complex cells. The unified nature of the adaption scheme allows us to construct a multilayered and task-independent attention selection network which uses the same learning rule for edge, motion, and color detection, and we can use this network to engage in attention selection in both static and dynamic scenes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Large Scale Nonlinear Control System Fine-Tuning Through Learning

    Page(s): 1009 - 1023
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1191 KB) |  | HTML iconHTML  

    Despite the continuous advances in the fields of intelligent control and computing, the design and deployment of efficient large scale nonlinear control systems (LNCSs) requires a tedious fine-tuning of the LNCS parameters before and during the actual system operation. In the majority of LNCSs the fine-tuning process is performed by experienced personnel based on field observations via experimentation with different combinations of controller parameters, without the use of a systematic approach. The existing adaptive/neural/fuzzy control methodologies cannot be used towards the development of a systematic, automated fine-tuning procedure for general LNCS due to the strict assumptions they impose on the controlled system dynamics; on the other hand, adaptive optimization methodologies fail to guarantee an efficient and safe performance during the fine-tuning process, mainly due to the fact that these methodologies involve the use of random perturbations. In this paper, we introduce and analyze, both by means of mathematical arguments and simulation experiments, a new learning/adaptive algorithm that can provide with convergent, an efficient and safe fine-tuning of general LNCS. The proposed algorithm consists of a combination of two different algorithms proposed by Kosmatopoulos (2007 and 2008) and the incremental-extreme learning machine neural networks (I-ELM-NNs). Among the nice properties of the proposed algorithm is that it significantly outperforms the algorithms proposed by Kosmatopoulos as well as other existing adaptive optimization algorithms. Moreover, contrary to the algorithms proposed by Kosmatopoulos , the proposed algorithm can operate efficiently in the case where the exogenous system inputs (e.g., disturbances, commands, demand, etc.) are unbounded signals. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Subgradient-Based Neural Networks for Nonsmooth Nonconvex Optimization Problems

    Page(s): 1024 - 1038
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (557 KB) |  | HTML iconHTML  

    This paper presents a subgradient-based neural network to solve a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function, a class of affine equality constraints, and a class of nonsmooth convex inequality constraints. The proposed neural network is modeled with a differential inclusion. Under a suitable assumption on the constraint set and a proper assumption on the objective function, it is proved that for a sufficiently large penalty parameter, there exists a unique global solution to the neural network and the trajectory of the network can reach the feasible region in finite time and stay there thereafter. It is proved that the trajectory of the neural network converges to the set which consists of the equilibrium points of the neural network, and coincides with the set which consists of the critical points of the objective function in the feasible region. A condition is given to ensure the convergence to the equilibrium point set in finite time. Moreover, under suitable assumptions, the coincidence between the solution to the differential inclusion and the ldquoslow solutionrdquo of it is also proved. Furthermore, three typical examples are given to present the effectiveness of the theoretic results obtained in this paper and the good performance of the proposed neural network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Growing and Pruning Method for Radial Basis Function Networks

    Page(s): 1039 - 1045
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (425 KB) |  | HTML iconHTML  

    A recently published generalized growing and pruning (GGAP) training algorithm for radial basis function (RBF) neural networks is studied and modified. GGAP is a resource-allocating network (RAN) algorithm, which means that a created network unit that consistently makes little contribution to the network's performance can be removed during the training. GGAP states a formula for computing the significance of the network units, which requires a d-fold numerical integration for arbitrary probability density function p(x) of the input data x (x isin R d) . In this work, the GGAP formula is approximated using a Gaussian mixture model (GMM) for p(x) and an analytical solution of the approximated unit significance is derived. This makes it possible to employ the modified GGAP for input data having complex and high-dimensional p(x), which was not possible in the original GGAP. The results of an extensive experimental study show that the modified algorithm outperforms the original GGAP achieving both a lower prediction error and reduced complexity of the trained network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Class of Sparsely Connected Autoassociative Morphological Memories for Large Color Images

    Page(s): 1045 - 1050
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (653 KB) |  | HTML iconHTML  

    This brief introduces a new class of sparsely connected autoassociative morphological memories (AMMs) that can be effectively used to process large multivalued patterns, which include color images as a particular case. Such as the single-valued AMMs, the multivalued models exhibit optimal absolute storage capacity and one-step convergence. The remarkable feature of the proposed models is their sparse structure. In fact, the number of synaptic junctions - and consequently the required computational resources - usually decreases considerably as more and more patterns are stored in the novel multivalued AMMs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Boundedness and Convergence of Online Gradient Method With Penalty for Feedforward Neural Networks

    Page(s): 1050 - 1054
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (216 KB) |  | HTML iconHTML  

    In this brief, we consider an online gradient method with penalty for training feedforward neural networks. Specifically, the penalty is a term proportional to the norm of the weights. Its roles in the method are to control the magnitude of the weights and to improve the generalization performance of the network. By proving that the weights are automatically bounded in the network training with penalty, we simplify the conditions that are required for convergence of online gradient method in literature. A numerical example is given to support the theoretical analysis. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Convergent Hybrid Decomposition Algorithm Model for SVM Training

    Page(s): 1055 - 1060
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (256 KB) |  | HTML iconHTML  

    Training of support vector machines (SVMs) requires to solve a linearly constrained convex quadratic problem. In real applications, the number of training data may be very huge and the Hessian matrix cannot be stored. In order to take into account this issue, a common strategy consists in using decomposition algorithms which at each iteration operate only on a small subset of variables, usually referred to as the working set. Training time can be significantly reduced by using a caching technique that allocates some memory space to store the columns of the Hessian matrix corresponding to the variables recently updated. The convergence properties of a decomposition method can be guaranteed by means of a suitable selection of the working set and this can limit the possibility of exploiting the information stored in the cache. We propose a general hybrid algorithm model which combines the capability of producing a globally convergent sequence of points with a flexible use of the information in the cache. As an example of a specific realization of the general hybrid model, we describe an algorithm based on a particular strategy for exploiting the information deriving from a caching technique. We report the results of computational experiments performed by simple implementations of this algorithm. The numerical results point out the potentiality of the approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • IEEE Computational Intelligence Society Information

    Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • Blank page [back cover]

    Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (5 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope