By Topic

Neural Networks, IEEE Transactions on

Issue 2 • Date Feb. 2010

Filter Results

Displaying Results 1 - 22 of 22
  • Table of contents

    Publication Year: 2010 , Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (36 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Publication Year: 2010 , Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • A Multiobjective Simultaneous Learning Framework for Clustering and Classification

    Publication Year: 2010 , Page(s): 185 - 200
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1889 KB) |  | HTML iconHTML  

    Traditional pattern recognition involves two tasks: clustering learning and classification learning. Clustering result can enhance the generalization ability of classification learning, while the class information can improve the accuracy of clustering learning. Hence, both learning methods can complement each other. To fuse the advantages of both learning methods together, many existing algorithms have been developed in a sequential fusing way by first optimizing the clustering criterion and then the classification criterion associated with the obtained clustering results. However, such kind of algorithms naturally fails to achieve the simultaneous optimality for two criteria, and thus has to sacrifice either the clustering performance or the classification performance. To overcome that problem, in this paper, we present a multiobjective simultaneous learning framework (MSCC) for both clustering and classification learning. MSCC utilizes multiple objective functions to formulate the clustering and classification problems, respectively, and more importantly, it employs the Bayesian theory to make these functions all only dependent on a set of the same parameters, i.e., clustering centers which play a role of the bridge connecting the clustering and classification learning. By simultaneously optimizing the clustering centers embedded in these functions, not only the effective clustering performance but also the promising classification performance can be simultaneously attained. Furthermore, from the multiple Pareto-optimality solutions obtained in MSCC, we can get an interesting observation that there is complementarity to great extent between clustering and classification learning processes. Empirical results on both synthetic and real data sets demonstrate the effectiveness and potential of MSCC. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Feature Extraction Using Constrained Approximation and Suppression

    Publication Year: 2010 , Page(s): 201 - 210
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (329 KB) |  | HTML iconHTML  

    In this paper, we systematize a family of constrained quadratic classifiers that belong to the class of one-class classifiers. One-class classifiers such as the single-class support vector machine or the subspace methods are widely used for pattern classification and detection problems because they have many advantages over binary classifiers. We interpret subspace methods as rank-constrained quadratic classifiers in the framework. We also introduce two constraints and a method of suppressing the effect of competing classes to make them more accurate and retain their advantages over binary classifiers. Experimental results demonstrate the advantages of our methods over conventional classifiers. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Growing Self-Reconstruction Maps

    Publication Year: 2010 , Page(s): 211 - 223
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4067 KB) |  | HTML iconHTML  

    In this paper, we propose a new method for surface reconstruction based on growing self-organizing maps (SOMs), called growing self-reconstruction maps (GSRMs). GSRM is an extension of growing neural gas (GNG) that includes the concept of triangular faces in the learning algorithm and additional conditions in order to include and remove connections, so that it can produce a triangular two-manifold mesh representation of a target object given an unstructured point cloud of its surface. The main modifications concern competitive Hebbian learning (CHL), the vertex insertion operation, and the edge removal mechanism. The method proposed is able to learn the geometry and topology of the surface represented in the point cloud and to generate meshes with different resolutions. Experimental results show that the proposed method can produce models that approximate the shape of an object, including its concave regions, boundaries, and holes, if any. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Design of the Inverse Function Delayed Neural Network for Solving Combinatorial Optimization Problems

    Publication Year: 2010 , Page(s): 224 - 237
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (913 KB) |  | HTML iconHTML  

    We have already proposed the inverse function delayed (ID) model as a novel neuron model. The ID model has a negative resistance similar to Bonhoeffer-van der Pol (BVP) model and the network has an energy function similar to Hopfield model. The neural network having an energy can converge on a solution of the combinatorial optimization problem and the computation is in parallel and hence fast. However, the existence of local minima is a serious problem. The negative resistance of the ID model can make the network state free from such local minima by selective destabilization. Hence, we expect that it has a potential to overcome the local minimum problems. In computer simulations, we have already shown that the ID network can be free from local minima and that it converges on the optimal solutions. However, the theoretical analysis has not been presented yet. In this paper, we redefine three types of constraints for the particular problems, then we analytically estimate the appropriate network parameters giving the global minimum states only. Moreover, we demonstrate the validity of estimated network parameters by computer simulations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cutting Plane Method for Continuously Constrained Kernel-Based Regression

    Publication Year: 2010 , Page(s): 238 - 247
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1081 KB) |  | HTML iconHTML  

    Incorporating constraints into the kernel-based regression is an effective means to improve regression performance. Nevertheless, in many applications, the constraints are continuous with respect to some parameters so that computational difficulties arise. Discretizing the constraints is a reasonable solution for these difficulties. However, in the context of kernel-based regression, most of existing works utilize the prior discretization strategy; this strategy suffers from a few inherent deficiencies: it cannot ensure that the regression result totally fulfills the original constraints and can hardly tackle high-dimensional problems. This paper proposes a cutting plane method (CPM) for constrained kernel-based regression problems and a relaxed CPM (R-CPM) for high-dimensional problems. The CPM discretizes the continuous constraints iteratively and ensures that the regression result strictly fulfills the original constraints. For high-dimensional problems, the R-CPM accepts a slight and controlled violation to attain a dimensional-independent computational complexity. The validity of the proposed methods is verified by numerical experiments. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust Independent Component Analysis by Iterative Maximization of the Kurtosis Contrast With Algebraic Optimal Step Size

    Publication Year: 2010 , Page(s): 248 - 261
    Cited by:  Papers (29)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (705 KB) |  | HTML iconHTML  

    Independent component analysis (ICA) aims at decomposing an observed random vector into statistically independent variables. Deflation-based implementations, such as the popular one-unit FastICA algorithm and its variants, extract the independent components one after another. A novel method for deflationary ICA, referred to as RobustICA, is put forward in this paper. This simple technique consists of performing exact line search optimization of the kurtosis contrast function. The step size leading to the global maximum of the contrast along the search direction is found among the roots of a fourth-degree polynomial. This polynomial rooting can be performed algebraically, and thus at low cost, at each iteration. Among other practical benefits, RobustICA can avoid prewhitening and deals with real- and complex-valued mixtures of possibly noncircular sources alike. The absence of prewhitening improves asymptotic performance. The algorithm is robust to local extrema and shows a very high convergence speed in terms of the computational cost required to reach a given source extraction quality, particularly for short data records. These features are demonstrated by a comparative numerical analysis on synthetic data. RobustICA's capabilities in processing real-world data involving noncircular complex strongly super-Gaussian sources are illustrated by the biomedical problem of atrial activity (AA) extraction in atrial fibrillation (AF) electrocardiograms (ECGs), where it outperforms an alternative ICA-based technique. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recursive Bayesian Recurrent Neural Networks for Time-Series Modeling

    Publication Year: 2010 , Page(s): 262 - 274
    Cited by:  Papers (19)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (780 KB) |  | HTML iconHTML  

    This paper develops a probabilistic approach to recursive second-order training of recurrent neural networks (RNNs) for improved time-series modeling. A general recursive Bayesian Levenberg-Marquardt algorithm is derived to sequentially update the weights and the covariance (Hessian) matrix. The main strengths of the approach are a principled handling of the regularization hyperparameters that leads to better generalization, and stable numerical performance. The framework involves the adaptation of a noise hyperparameter and local weight prior hyperparameters, which represent the noise in the data and the uncertainties in the model parameters. Experimental investigations using artificial and real-world data sets show that RNNs equipped with the proposed approach outperform standard real-time recurrent learning and extended Kalman training algorithms for recurrent networks, as well as other contemporary nonlinear neural models, on time-series modeling. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Hierarchical RBF Online Learning Algorithm for Real-Time 3-D Scanner

    Publication Year: 2010 , Page(s): 275 - 285
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1386 KB) |  | HTML iconHTML  

    In this paper, a novel real-time online network model is presented. It is derived from the hierarchical radial basis function (HRBF) model and it grows by automatically adding units at smaller scales, where the surface details are located, while data points are being collected. Real-time operation is achieved by exploiting the quasi-local nature of the Gaussian units: through the definition of a quad-tree structure to support their receptive field local network reconfiguration can be obtained. The model has been applied to 3-D scanning, where an updated real-time display of the manifold to the operator is fundamental to drive the acquisition procedure itself. Quantitative results are reported, which show that the accuracy achieved is comparable to that of two batch approaches: batch HRBF and support vector machines (SVMs). However, these two approaches are not suitable to real-time online learning. Moreover, proof of convergence is also given. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Large Developing Receptive Fields Using a Distributed and Locally Reprogrammable Address–Event Receiver

    Publication Year: 2010 , Page(s): 286 - 304
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1609 KB) |  | HTML iconHTML  

    A distributed and locally reprogrammable address-event receiver has been designed, in which incoming address-events are monitored simultaneously by all synapses, allowing for arbitrarily large axonal fan-out without reducing channel capacity. Synapses can change the address of their presynaptic neuron, allowing the distributed implementation of a biologically realistic learning rule, with both synapse formation and elimination (synaptic rewiring). Probabilistic synapse formation leads to topographic map development, made possible by a cross-chip current-mode calculation of Euclidean distance. As well as synaptic plasticity in rewiring, synapses change weights using a competitive Hebbian learning rule (spike-timing-dependent plasticity). The weight plasticity allows receptive fields to be modified based on spatio-temporal correlations in the inputs, and the rewiring plasticity allows these modifications to become embedded in the network topology. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis of the Distance Between Two Classes for Tuning SVM Hyperparameters

    Publication Year: 2010 , Page(s): 305 - 318
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1374 KB) |  | HTML iconHTML  

    An important step in the construction of a support vector machine (SVM) is to select optimal hyperparameters. This paper proposes a novel method for tuning the hyperparameters by maximizing the distance between two classes (DBTC) in the feature space. With a normalized kernel function, we find that DBTC can be used as a class separability criterion since the between-class separation and the within-class data distribution are implicitly taken into account. Employing DBTC as an objective function, we develop a gradient-based algorithm to search the optimal kernel parameter. On the basis of the geometric analysis and simulation results, we find that the optimal algorithm and the initialization problem become very simple. Experimental results on the synthetic and real-world data show that the proposed method consistently outperforms other existing hyperparameter tuning methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Linear Time Maximum Margin Clustering

    Publication Year: 2010 , Page(s): 319 - 332
    Cited by:  Papers (19)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1168 KB) |  | HTML iconHTML  

    Maximum margin clustering (MMC) is a newly proposed clustering method which has shown promising performance in recent studies. It extends the computational techniques of support vector machine (SVM) to the unsupervised scenario. Traditionally, MMC is formulated as a nonconvex integer programming problem which makes it difficult to solve. Several methods have been proposed in the literature to solve the MMC problem based on either semidefinite programming (SDP) or alternating optimization. However, these methods are still time demanding when handling large scale data sets, which limits its application in real-world problems. In this paper, we propose a cutting plane maximum margin clustering (CPMMC) algorithm. It first decomposes the nonconvex MMC problem into a series of convex subproblems by making use of the constrained concave-convex procedure (CCCP), then for each subproblem, our algorithm adopts the cutting plane algorithm to solve it. Moreover, we show that the CPMMC algorithm takes O(sn) time to converge with guaranteed accuracy, where n is the number of samples in the data set and s is the sparsity of the data set, i.e., the average number of nonzero features of the data samples. We also derive the multiclass version of our CPMMC algorithm. Experimental evaluations on several real-world data sets show that CPMMC performs better than existing MMC methods, both in efficiency and accuracy. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Representation of a Fisher Criterion Function in a Kernel Feature Space

    Publication Year: 2010 , Page(s): 333 - 339
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (419 KB) |  | HTML iconHTML  

    In this brief, we consider kernel methods for classification (Shawe-Taylor and Cristianini, 2004) from a separability point of view and provide a representation of the Fisher criterion function in a kernel feature space. We then show that the value of the Fisher function can be simply computed by using averages of diagonal and off-diagonal blocks of a kernel matrix. This result further serves to reveal that the ideal kernel matrix is a global solution to the problem of maximizing the Fisher criterion function. Its relation to an empirical kernel target alignment is then reported. To demonstrate the usefulness of these theories, we provide an application study for classification of prostate cancer based on microarray data sets. The results show that the parameter of a kernel function can be readily optimized. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A New Method for Stability Analysis of Recurrent Neural Networks With Interval Time-Varying Delay

    Publication Year: 2010 , Page(s): 339 - 344
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (248 KB) |  | HTML iconHTML  

    This brief deals with the problem of stability analysis for a class of recurrent neural networks (RNNs) with a time-varying delay in a range. Both delay-independent and delay-dependent conditions are derived. For the former, an augmented Lyapunov functional is constructed and the derivative of the state is retained. Since the obtained criterion realizes the decoupling of the Lyapunov function matrix and the coefficient matrix of the neural networks, it can be easily extended to handle neural networks with polytopic uncertainties. For the latter, a new type of delay-range-dependent condition is proposed using the free-weighting matrix technique to obtain a tighter upper bound on the derivative of the Lyapunov-Krasovskii functional. Two examples are given to illustrate the effectiveness and the reduced conservatism of the proposed results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive Tracking for Periodically Time-Varying and Nonlinearly Parameterized Systems Using Multilayer Neural Networks

    Publication Year: 2010 , Page(s): 345 - 351
    Cited by:  Papers (35)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (328 KB) |  | HTML iconHTML  

    This brief addresses the problem of designing adaptive neural network tracking control for a class of strict-feedback systems with unknown time-varying disturbances of known periods which nonlinearly appear in unknown functions. Multilayer neural network (MNN) and Fourier series expansion (FSE) are combined into a novel approximator to model each uncertainty in systems. Dynamic surface control (DSC) approach and integral-type Lyapunov function (ILF) technique are combined to design the control algorithm. The ultimate uniform boundedness of all closed-loop signals is guaranteed. The tracking error is proved to converge to a small residual set around the origin. Two simulation examples are provided to illustrate the feasibility of control scheme proposed in this brief. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Scaling Up Support Vector Machines Using Nearest Neighbor Condensation

    Publication Year: 2010 , Page(s): 351 - 357
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (586 KB) |  | HTML iconHTML  

    In this brief, we describe the FCNN-SVM classifier, which combines the support vector machine (SVM) approach and the fast nearest neighbor condensation classification rule (FCNN) in order to make SVMs practical on large collections of data. As a main contribution, it is experimentally shown that, on very large and multidimensional data sets, the FCNN-SVM is one or two orders of magnitude faster than SVM, and that the number of support vectors (SVs) is more than halved with respect to SVM. Thus, a drastic reduction of both training and testing time is achieved by using the FCNN-SVM. This result is obtained at the expense of a little loss of accuracy. The FCNN-SVM is proposed as a viable alternative to the standard SVM in applications where a fast response time is a fundamental requirement. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Special issue on white box nonlinear prediction models

    Publication Year: 2010 , Page(s): 358
    Save to Project icon | Request Permissions | PDF file iconPDF (151 KB)  
    Freely Available from IEEE
  • 2010 IEEE World Congress on Computational Intelligence (WCCI)

    Publication Year: 2010 , Page(s): 359
    Save to Project icon | Request Permissions | PDF file iconPDF (754 KB)  
    Freely Available from IEEE
  • Proven powerful [advertisement]

    Publication Year: 2010 , Page(s): 360
    Save to Project icon | Request Permissions | PDF file iconPDF (188 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Publication Year: 2010 , Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks Information for authors

    Publication Year: 2010 , Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope