Scheduled System Maintenance:
Some services will be unavailable Sunday, March 29th through Monday, March 30th. We apologize for the inconvenience.
By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 2006

Filter Results

Displaying Results 1 - 25 of 37
  • Table of contents

    Publication Year: 2006 , Page(s): C1 - C4
    Save to Project icon | Request Permissions | PDF file iconPDF (42 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Publication Year: 2006 , Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (36 KB)  
    Freely Available from IEEE
  • Evaluation of Neural Network Robust Reliability Using Information-Gap Theory

    Publication Year: 2006 , Page(s): 1349 - 1361
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (764 KB) |  | HTML iconHTML  

    A novel technique for the evaluation of neural network robustness against uncertainty using a nonprobabilistic approach is presented. Conventional optimization techniques were employed to train multilayer perceptron (MLP) networks, which were then probed with an uncertainty analysis using an information-gap model to quantify the network response to uncertainty in the input data. It is demonstrated that the best performing network on data with low uncertainty is not in general the optimal network on data with a higher degree of input uncertainty. Using the concepts of information-gap theory, this paper develops a theoretical framework for information-gap uncertainty applied to neural networks, and explores the practical application of the procedure to three sample cases. The first consists of a simple two-dimensional (2-D) classification network operating on a known Gaussian distribution, the second a nine-lass vibration classification problem from an aircraft wing, and the third a two-class example from a database of breast cancer incidence View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global Convergence of Decomposition Learning Methods for Support Vector Machines

    Publication Year: 2006 , Page(s): 1362 - 1369
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (364 KB) |  | HTML iconHTML  

    Decomposition methods are well-known techniques for solving quadratic programming (QP) problems arising in support vector machines (SVMs). In each iteration of a decomposition method, a small number of variables are selected and a QP problem with only the selected variables is solved. Since large matrix computations are not required, decomposition methods are applicable to large QP problems. In this paper, we will make a rigorous analysis of the global convergence of general decomposition methods for SVMs. We first introduce a relaxed version of the optimality condition for the QP problems and then prove that a decomposition method reaches a solution satisfying this relaxed optimality condition within a finite number of iterations under a very mild condition on how to select variables View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The FastICA Algorithm Revisited: Convergence Analysis

    Publication Year: 2006 , Page(s): 1370 - 1381
    Cited by:  Papers (21)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (392 KB) |  | HTML iconHTML  

    The fast independent component analysis (FastICA) algorithm is one of the most popular methods to solve problems in ICA and blind source separation. It has been shown experimentally that it outperforms most of the commonly used ICA algorithms in convergence speed. A rigorous local convergence analysis has been presented only for the so-called one-unit case, in which just one of the rows of the separating matrix is considered. However, in the FastICA algorithm, there is also an explicit normalization step, and it may be questioned whether the extra rotation caused by the normalization will affect the convergence speed. The purpose of this paper is to show that this is not the case and the good convergence properties of the one-unit case are also shared by the full algorithm with symmetrical normalization. A local convergence analysis is given for the general case, and the global behavior is illustrated numerically for two sources and two mixtures in several typical cases View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Locally Weighted Interpolating Growing Neural Gas

    Publication Year: 2006 , Page(s): 1382 - 1393
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1112 KB) |  | HTML iconHTML  

    In this paper, we propose a new approach to function approximation based on a growing neural gas (GNG), a self-organizing map (SOM) which is able to adapt to the local dimension of a possible high-dimensional input distribution. Local models are built interpolating between values associated with the map's neurons. These models are combined using a weighted sum to yield the final approximation value. The values, the positions, and the "local ranges" of the neurons are adapted to improve the approximation quality. The method is able to adapt to changing target functions and to follow nonstationary input distributions. The new approach is compared to the radial basis function (RBF) extension of the growing neural gas and to locally weighted projection regression (LWPR), a state-of-the-art algorithm for incremental nonlinear function approximation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • FCMAC-Yager: A Novel Yager-Inference-Scheme-Based Fuzzy CMAC

    Publication Year: 2006 , Page(s): 1394 - 1410
    Cited by:  Papers (23)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1328 KB) |  | HTML iconHTML  

    The cerebellum is a brain region important for a number of motor and cognitive functions. It is able to generate error correction signals to drive learning and for the acquisition of memory skills. The cerebellar model articulation controller (CMAC) is a neural network inspired by the neurophysiologic theory of the cerebellum and is recognized for its localized generalization and rapid algorithmic computation capabilities. The main deficiencies in the basic CMAC structure are: 1) it is difficult to interpret the internal operations of the CMAC network and 2) the resolution (quantization) problem arising from the partitioning of the input training space. These limitations lead to the synthesis of a fuzzy quantization technique and the mapping of a fuzzy inference scheme onto the CMAC structure. The discrete incremental clustering (DIC) technique is employed to alleviate the quantization problem in the CMAC structure, resulting in the fuzzy CMAC (FCMAC) network. The Yager inference scheme (Yager), which possesses firm fuzzy logic foundation and maps closely to the logical implication operations in the classical (binary) logic framework, is subsequently mapped onto the FCMAC structure. This results in a novel fuzzy neural architecture known as the fuzzy cerebellar model articulation controller-Yager (FCMAC-Yager) system. The proposed FCMAC-Yager network exhibits learning and memory capabilities of the cerebellum through the CMAC structure while emulating the human way of reasoning through the Yager. The new FCMAC-Yager network employs a two-phase training algorithm consisting of structural learning based on the DIC technique and parameter learning using hebbian learning (associative long-term potentiation). The proposed FCMAC-Yager architecture is evaluated using an extensive suite of real-life applications such as highway traffic-trend modeling and prediction and performing as an early warning system for bank failure classification and medical diagnosis of breast can- er. The experimental results are encouraging View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks

    Publication Year: 2006 , Page(s): 1411 - 1423
    Cited by:  Papers (97)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (673 KB) |  | HTML iconHTML  

    In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Parameter Incremental Learning Algorithm for Neural Networks

    Publication Year: 2006 , Page(s): 1424 - 1438
    Cited by:  Papers (21)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (977 KB) |  | HTML iconHTML  

    In this paper, a novel stochastic (or online) training algorithm for neural networks, named parameter incremental learning (PIL) algorithm, is proposed and developed. The main idea of the PIL strategy is that the learning algorithm should not only adapt to the newly presented input-output training pattern by adjusting parameters, but also preserve the prior results. A general PIL algorithm for feedforward neural networks is accordingly presented as the first-order approximate solution to an optimization problem, where the performance index is the combination of proper measures of preservation and adaptation. The PIL algorithms for the multilayer perceptron (MLP) are subsequently derived. Numerical studies show that for all the three benchmark problems used in this paper the PIL algorithm for MLP is measurably superior to the standard online backpropagation (BP) algorithm and the stochastic diagonal Levenberg-Marquardt (SDLM) algorithm in terms of the convergence speed and accuracy. Other appealing features of the PIL algorithm are that it is computationally as simple as the BP algorithm, and as easy to use as the BP algorithm. It, therefore, can be applied, with better performance, to any situations where the standard online BP algorithm is applicable View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Hybrid Forward Algorithm for RBF Neural Network Construction

    Publication Year: 2006 , Page(s): 1439 - 1451
    Cited by:  Papers (15)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (581 KB) |  | HTML iconHTML  

    This paper proposes a novel hybrid forward algorithm (HFA) for the construction of radial basis function (RBF) neural networks with tunable nodes. The main objective is to efficiently and effectively produce a parsimonious RBF neural network that generalizes well. In this study, it is achieved through simultaneous network structure determination and parameter optimization on the continuous parameter space. This is a mixed integer hard problem and the proposed HFA tackles this problem using an integrated analytic framework, leading to significantly improved network performance and reduced memory usage for the network construction. The computational complexity analysis confirms the efficiency of the proposed algorithm, and the simulation results demonstrate its effectiveness View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An Optimization Methodology for Neural Network Weights and Architectures

    Publication Year: 2006 , Page(s): 1452 - 1459
    Cited by:  Papers (25)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (329 KB) |  | HTML iconHTML  

    This paper introduces a methodology for neural network global optimization. The aim is the simultaneous optimization of multilayer perceptron (MLP) network weights and architectures, in order to generate topologies with few connections and high classification performance for any data sets. The approach combines the advantages of simulated annealing, tabu search and the backpropagation training algorithm in order to generate an automatic process for producing networks with high classification performance and low complexity. Experimental results obtained with four classification problems and one prediction problem has shown to be better than those obtained by the most commonly used optimization techniques View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Associative Learning in Hierarchical Self-Organizing Learning Arrays

    Publication Year: 2006 , Page(s): 1460 - 1470
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1047 KB) |  | HTML iconHTML  

    In this paper, we introduce feedback-based associative learning in self-organized learning arrays (SOLAR). SOLAR structures are hierarchically organized networks of sparsely connected neurons that define their own functions and select their interconnections locally. This paper provides a description of neuron self-organization and signal processing. Feedforward processing is used to make necessary correlations and learn the input patterns. Discovered associations between neuron inputs are used to generate feedback signals. These feedback signals, when propagated to the primary inputs, can establish the expected input values. This can be used for heteroassociative (HA) and autoassociative (AA) learning and pattern recognition. Example applications in HA learning are given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Convergence of Neural Networks for Programming Problems via a Nonsmooth Łojasiewicz Inequality

    Publication Year: 2006 , Page(s): 1471 - 1486
    Cited by:  Papers (23)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (593 KB) |  | HTML iconHTML  

    This paper considers a class of neural networks (NNs) for solving linear programming (LP) problems, convex quadratic programming (QP) problems, and nonconvex QP problems where an indefinite quadratic objective function is subject to a set of affine constraints. The NNs are characterized by constraint neurons modeled by ideal diodes with vertical segments in their characteristic, which enable to implement an exact penalty method. A new method is exploited to address convergence of trajectories, which is based on a nonsmooth Lstrokojasiewicz inequality for the generalized gradient vector field describing the NN dynamics. The method permits to prove that each forward trajectory of the NN has finite length, and as a consequence it converges toward a singleton. Furthermore, by means of a quantitative evaluation of the Lstrokojasiewicz exponent at the equilibrium points, the following results on convergence rate of trajectories are established: 1) for nonconvex QP problems, each trajectory is either exponentially convergent, or convergent in finite time, toward a singleton belonging to the set of constrained critical points; 2) for convex QP problems, the same result as in 1) holds; moreover, the singleton belongs to the set of global minimizers; and 3) for LP problems, each trajectory converges in finite time to a singleton belonging to the set of global minimizers. These results, which improve previous results obtained via the Lyapunov approach, are true independently of the nature of the set of equilibrium points, and in particular they hold even when the NN possesses infinitely many nonisolated equilibrium points View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network

    Publication Year: 2006 , Page(s): 1487 - 1499
    Cited by:  Papers (36)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (471 KB) |  | HTML iconHTML  

    In recent years, a recurrent neural network called projection neural network was proposed for solving monotone variational inequalities and related convex optimization problems. In this paper, we show that the projection neural network can also be used to solve pseudomonotone variational inequalities and related pseudoconvex optimization problems. Under various pseudomonotonicity conditions and other conditions, the projection neural network is proved to be stable in the sense of Lyapunov and globally convergent, globally asymptotically stable, and globally exponentially stable. Since monotonicity is a special case of pseudomononicity, the projection neural network can be applied to solve a broader class of constrained optimization problems related to variational inequalities. Moreover, a new concept, called componentwise pseudomononicity, different from pseudomononicity in general, is introduced. Under this new concept, two stability results of the projection neural network for solving variational inequalities are also obtained. Finally, numerical examples show the effectiveness and performance of the projection neural network View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application

    Publication Year: 2006 , Page(s): 1500 - 1510
    Cited by:  Papers (57)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (812 KB) |  | HTML iconHTML  

    The design, analysis, and application of a new recurrent neural network for quadratic programming, called simplified dual neural network, are discussed. The analysis mainly concentrates on the convergence property and the computational complexity of the neural network. The simplified dual neural network is shown to be globally convergent to the exact optimal solution. The complexity of the neural network architecture is reduced with the number of neurons equal to the number of inequality constraints. Its application to k-winners-take-all (KWTA) operation is discussed to demonstrate how to solve problems with this neural network View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural Networks for Continuous Online Learning and Control

    Publication Year: 2006 , Page(s): 1511 - 1531
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2647 KB) |  | HTML iconHTML  

    This paper proposes a new hybrid neural network (NN) model that employs a multistage online learning process to solve the distributed control problem with an infinite horizon. Various techniques such as reinforcement learning and evolutionary algorithm are used to design the multistage online learning process. For this paper, the infinite horizon distributed control problem is implemented in the form of real-time distributed traffic signal control for intersections in a large-scale traffic network. The hybrid neural network model is used to design each of the local traffic signal controllers at the respective intersections. As the state of the traffic network changes due to random fluctuation of traffic volumes, the NN-based local controllers will need to adapt to the changing dynamics in order to provide effective traffic signal control and to prevent the traffic network from becoming overcongested. Such a problem is especially challenging if the local controllers are used for an infinite horizon problem where online learning has to take place continuously once the controllers are implemented into the traffic network. A comprehensive simulation model of a section of the Central Business District (CBD) of Singapore has been developed using PARAMICS microscopic simulation program. As the complexity of the simulation increases, results show that the hybrid NN model provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as a new, continuously updated simultaneous perturbation stochastic approximation-based neural network (SPSA-NN). Using the hybrid NN model, the total mean delay of each vehicle has been reduced by 78% and the total mean stoppage time of each vehicle has been reduced by 84% compared to the existing traffic signal control algorithm. This shows the efficacy of the hybrid NN model in solving large-scale traffic signal control problem in a distributed manner. Also, it indicates the- possibility of using the hybrid NN model for other applications that are similar in nature as the infinite horizon distributed control problem View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Developmental Learning With Behavioral Mode Tuning by Carrier-Frequency Modulation in Coherent Neural Networks

    Publication Year: 2006 , Page(s): 1532 - 1543
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1555 KB) |  | HTML iconHTML  

    We propose a developmental learning architecture with which a motion-control system learns multiple tasks similar to each other or advanced ones incrementally and efficiently by tuning its behavioral mode. The system is based on a coherent neural network whose carrier frequency works as a mode-tuning parameter. In our experiments, we consider two tasks related to bicycle riding. The first is to ride as temporally long as the system can before it falls down (task 1). The second is an advanced one, i.e., to ride as far as possible in a certain direction (task 2). We compare developmental learning to learn task 2 after task 1 with the direct learning of task 2. We also examine the effect of the mode tuning by comparing variable-mode learning (VML), where the carrier frequency is set free to move, with fixed-mode learning (FML), where the frequency is unchanged. We find that VML developmental learning results in the most efficient learning among the possible combinations. We discuss the effects of the incremental task assignment as well as the behavioral mode tuning in developmental learning View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • One-Class-at-a-Time Removal Sequence Planning Method for Multiclass Classification Problems

    Publication Year: 2006 , Page(s): 1544 - 1549
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (574 KB) |  | HTML iconHTML  

    Using dynamic programming, this work develops a one-class-at-a-time removal sequence planning method to decompose a multiclass classification problem into a series of two-class problems. Compared with previous decomposition methods, the approach has the following distinct features. First, under the one-class-at-a-time framework, the approach guarantees the optimality of the decomposition. Second, for a K-class problem, the number of binary classifiers required by the method is only K-1. Third, to achieve higher classification accuracy, the approach can easily be adapted to form a committee machine. A drawback of the approach is that its computational burden increases rapidly with the number of classes. To resolve this difficulty, a partial decomposition technique is introduced that reduces the computational cost by generating a suboptimal solution. Experimental results demonstrate that the proposed approach consistently outperforms two conventional decomposition methods View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Discriminative Common Vector Method With Kernels

    Publication Year: 2006 , Page(s): 1550 - 1565
    Cited by:  Papers (17)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1412 KB) |  | HTML iconHTML  

    In some pattern recognition tasks, the dimension of the sample space is larger than the number of samples in the training set. This is known as the "small sample size problem". Linear discriminant analysis (LDA) techniques cannot be applied directly to the small sample size case. The small sample size problem is also encountered when kernel approaches are used for recognition. In this paper, we attempt to answer the question of "How should one choose the optimal projection vectors for feature extraction in the small sample size case?" Based on our findings, we propose a new method called the kernel discriminative common vector method. In this method, we first nonlinearly map the original input space to an implicit higher dimensional feature space, in which the data are hoped to be linearly separable. Then, the optimal projection vectors are computed in this transformed space. The proposed method yields an optimal solution for maximizing a modified Fisher's linear discriminant criterion, discussed in the paper. Thus, under certain conditions, a 100% recognition rate is guaranteed for the training set samples. Experiments on test data also show that, in many situations, the generalization performance of the proposed method compares favorably with other kernel approaches View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Permutation Coding Technique for Image Recognition Systems

    Publication Year: 2006 , Page(s): 1566 - 1579
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1777 KB) |  | HTML iconHTML  

    A feature extractor and neural classifier for image recognition systems are proposed. The proposed feature extractor is based on the concept of random local descriptors (RLDs). It is followed by the encoder that is based on the permutation coding technique that allows to take into account not only detected features but also the position of each feature on the image and to make the recognition process invariant to small displacements. The combination of RLDs and permutation coding permits us to obtain a sufficiently general description of the image to be recognized. The code generated by the encoder is used as an input data for the neural classifier. Different types of images were used to test the proposed image recognition system. It was tested in the handwritten digit recognition problem, the face recognition problem, and the microobject shape recognition problem. The results of testing are very promising. The error rate for the Modified National Institute of Standards and Technology (MNIST) database is 0.44% and for the Olivetti Research Laboratory (ORL) database it is 0.1% View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A New Adaptive Backpropagation Algorithm Based on Lyapunov Stability Theory for Neural Networks

    Publication Year: 2006 , Page(s): 1580 - 1591
    Cited by:  Papers (27)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1019 KB) |  | HTML iconHTML  

    A new adaptive backpropagation (BP) algorithm based on Lyapunov stability theory for neural networks is developed in this paper. It is shown that the candidate of a Lyapunov function V(k) of the tracking error between the output of a neural network and the desired reference signal is chosen first, and the weights of the neural network are then updated, from the output layer to the input layer, in the sense that DeltaV(k)=V(k)-V(k-1)<0. The output tracking error can then asymptotically converge to zero according to Lyapunov stability theory. Unlike gradient-based BP training algorithms, the new Lyapunov adaptive BP algorithm in this paper is not used for searching the global minimum point along the cost-function surface in the weight space, but it is aimed at constructing an energy surface with a single global minimum point through the adaptive adjustment of the weights as the time goes to infinity. Although a neural network may have bounded input disturbances, the effects of the disturbances can be eliminated, and asymptotic error convergence can be obtained. The new Lyapunov adaptive BP algorithm is then applied to the design of an adaptive filter in the simulation example to show the fast error convergence and strong robustness with respect to large bounded input disturbances View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiuser Receiver for DS-CDMA Signals in Multipath Channels: An Enhanced Multisurface Method

    Publication Year: 2006 , Page(s): 1592 - 1605
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (567 KB) |  | HTML iconHTML  

    This paper deals with the problem of multiuser detection in direct-sequence code-division multiple-access (DS-CDMA) systems in multipath environments. The existing multiuser detectors can be divided into two categories: 1) low-complexity poor-performance linear detectors and 2) high-complexity good-performance nonlinear detectors. In particular, in channels where the orthogonality of the code sequences is destroyed by multipath, detectors with linear complexity perform much worse than the nonlinear detectors. In this paper, we propose an enhanced multisurface method (EMSM) for multiuser detection in multipath channels. EMSM is an intermediate piecewise linear detection scheme with a run-time complexity linear in the number of users. Its bit error rate performance is compared with existing linear detectors, a nonlinear radial basis function detector trained by the new support vector learning algorithm, and Verdu's optimal detector. Simulations in multipath channels, for both synchronous and asynchronous cases, indicate that it always outperforms all other linear detectors, performing nearly as well as nonlinear detectors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Training Recurrent Neurocontrollers for Robustness With Derivative-Free Kalman Filter

    Publication Year: 2006 , Page(s): 1606 - 1616
    Cited by:  Papers (8)  |  Patents (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (477 KB) |  | HTML iconHTML  

    We are interested in training neurocontrollers for robustness on discrete-time models of physical systems. Our neurocontrollers are implemented as recurrent neural networks (RNNs). A model of the system to be controlled is known to the extent of parameters and/or signal uncertainties. Parameter values are drawn from a known distribution. For each instance of the model with specified parameters, a recurrent neurocontroller is trained by evaluating sensitivities of the model outputs to perturbations of the neurocontroller weights and incrementally updating the weights. Our training process strives to minimize a quadratic cost function averaged over many different models. In the end, the process yields a robust recurrent neurocontroller, which is ready for deployment with fixed weights. We employ a derivative-free Kalman filter algorithm proposed by Norgaard and extended by Feldkamp (2001) and Feldkamp (2002) to neural network training. Our training algorithm combines effectiveness of a second-order training method with universal applicability to both differentiable and nondifferentiable systems. Our approach is that of model reference control, and it extends significantly the capabilities proposed by Prokhorov (2001). We illustrate it with two examples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Support Vector Machines for Nonlinear Kernel ARMA System Identification

    Publication Year: 2006 , Page(s): 1617 - 1622
    Cited by:  Papers (19)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (251 KB) |  | HTML iconHTML  

    Nonlinear system identification based on support vector machines (SVM) has been usually addressed by means of the standard SVM regression (SVR), which can be seen as an implicit nonlinear autoregressive and moving average (ARMA) model in some reproducing kernel Hilbert space (RKHS). The proposal of this letter is twofold. First, the explicit consideration of an ARMA model in an RKHS (SVM-ARMA2K) is proposed. We show that stating the ARMA equations in an RKHS leads to solving the regularized normal equations in that RKHS, in terms of the autocorrelation and cross correlation of the (nonlinearly) transformed input and output discrete time processes. Second, a general class of SVM-based system identification nonlinear models is presented, based on the use of composite Mercer's kernels. This general class can improve model flexibility by emphasizing the input-output cross information (SVM-ARMA4K), which leads to straightforward and natural combinations of implicit and explicit ARMA models (SVR-ARMA2K and SVR-ARMA4K). Capabilities of these different SVM-based system identification schemes are illustrated with two benchmark problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimating the Number of Hidden Neurons in a Feedforward Network Using the Singular Value Decomposition

    Publication Year: 2006 , Page(s): 1623 - 1629
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2265 KB) |  | HTML iconHTML  

    In this letter, we attempt to quantify the significance of increasing the number of neurons in the hidden layer of a feedforward neural network architecture using the singular value decomposition (SVD). Through this, we extend some well-known properties of the SVD in evaluating the generalizability of single hidden layer feedforward networks (SLFNs) with respect to the number of hidden layer neurons. The generalization capability of the SLFN is measured by the degree of linear independency of the patterns in hidden layer space, which can be indirectly quantified from the singular values obtained from the SVD, in a postlearning step. A pruning/growing technique based on these singular values is then used to estimate the necessary number of neurons in the hidden layer. More importantly, we describe in detail properties of the SVD in determining the structure of a neural network particularly with respect to the robustness of the selected model View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope