By Topic

Neural Networks, IEEE Transactions on

Issue 4 • Date April 2010

Filter Results

Displaying Results 1 - 22 of 22
  • Table of contents

    Page(s): C1
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • An Adaptive Multiobjective Approach to Evolving ART Architectures

    Page(s): 529 - 550
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1226 KB) |  | HTML iconHTML  

    In this paper, we present the evolution of adaptive resonance theory (ART) neural network architectures (classifiers) using a multiobjective optimization approach. In particular, we propose the use of a multiobjective evolutionary approach to simultaneously evolve the weights and the topology of three well-known ART architectures; fuzzy ARTMAP (FAM), ellipsoidal ARTMAP (EAM), and Gaussian ARTMAP (GAM). We refer to the resulting architectures as MO-GFAM, MO-GEAM, and MO-GGAM, and collectively as MO-GART. The major advantage of MO-GART is that it produces a number of solutions for the classification problem at hand that have different levels of merit [accuracy on unseen data (generalization) and size (number of categories created)]. MO-GART is shown to be more elegant (does not require user intervention to define the network parameters), more effective (of better accuracy and smaller size), and more efficient (faster to produce the solution networks) than other ART neural network architectures that have appeared in the literature. Furthermore, MO-GART is shown to be competitive with other popular classifiers, such as classification and regression tree (CART) and support vector machines (SVMs). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Penalized Preimage Learning in Kernel Principal Component Analysis

    Page(s): 551 - 570
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (9766 KB) |  | HTML iconHTML  

    Finding the preimage of a feature vector in kernel principal component analysis (KPCA) is of crucial importance when KPCA is applied in some applications such as image preprocessing. Since the exact preimage of a feature vector in the kernel feature space, normally, does not exist in the input data space, an approximate preimage is learned and encouraging results have been reported in the last few years. However, it is still difficult to find a "good" estimation of preimage. As estimation of preimage in kernel methods is ill-posed, how to guide the preimage learning for a better estimation is important and still an open problem. To address this problem, a penalized strategy is developed in this paper, where some penalization terms are used to guide the preimage learning process. To develop an efficient penalized technique, we first propose a two-step general framework, in which a preimage is directly modeled by weighted combination of the observed samples and the weights are learned by some optimization function subject to certain constraints. Compared to existing techniques, this would also give advantages in directly turning preimage learning into the optimization of the combination weights. Under this framework, a penalized methodology is developed by integrating two types of penalizations. First, to ensure learning a well-defined preimage, of which each entry is not out of data range, convexity constraint is imposed for learning the combination weights. More insight effects of the convexity constraint are also explored. Second, a penalized function is integrated as part of the optimization function to guide the preimage learning process. Particularly, the weakly supervised penalty is proposed, discussed, and extensively evaluated along with Laplacian penalty and ridge penalty. It could be further interpreted that the learned preimage can preserve some kind of pointwise conditional mutual information. Finally, KPCA with preimage learning is applied on face image - - data sets in the aspects of facial expression normalization, face image denoising, recovery of missing parts from occlusion, and illumination normalization. Experimental results show that the proposed preimage learning algorithm obtains lower mean square error (MSE) and better visual quality of reconstructed images. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Exponential Synchronization of Hybrid Coupled Networks With Delayed Coupling

    Page(s): 571 - 583
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1042 KB) |  | HTML iconHTML  

    This paper investigates exponential synchronization of coupled networks with hybrid coupling, which is composed of constant coupling and discrete-delay coupling. There is only one transmittal delay in the delayed coupling. The fact is that in the signal transmission process, the time delay affects only the variable that is being transmitted from one system to another, then it makes sense to assume that there is only one single delay contributing to the dynamics. Some sufficient conditions for synchronization are derived based on Lyapunov functional and linear matrix inequality (LMI). In particular, the coupling matrix may be asymmetric or nondiagonal. Moreover, the transmittal delay can be different from the one in the isolated system. A distinctive feature of this work is that the synchronized state will vary in comparison with the conventional synchronized solution. Especially, the degree of the nodes and the inner delayed coupling matrix heavily influence the synchronized state. Finally, a chaotic neural network is used as the node in two regular networks to show the effectiveness of the proposed criteria. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • State–Space Analysis of Boolean Networks

    Page(s): 584 - 594
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (378 KB) |  | HTML iconHTML  

    This paper provides a comprehensive framework for the state-space approach to Boolean networks. First, it surveys the authors' recent work on the topic: Using semitensor product of matrices and the matrix expression of logic, the logical dynamic equations of Boolean (control) networks can be converted into standard discrete-time dynamics. To use the state-space approach, the state space and its subspaces of a Boolean network have been carefully defined. The basis of a subspace has been constructed. Particularly, the regular subspace, Y-friendly subspace, and invariant subspace are precisely defined, and the verifying algorithms are presented. As an application, the indistinct rolling gear structure of a Boolean network is revealed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Conformation-Based Hidden Markov Models: Application to Human Face Identification

    Page(s): 595 - 608
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (807 KB) |  | HTML iconHTML  

    Hidden Markov models (HMMs) and their variants are capable to classify complex and structured objects. However, one of their major restrictions is their inability to cope with shape or conformation intrinsically: HMM-based techniques have difficulty predicting the n-dimensional shape formed by the symbols of the visible observation (VO) sequence. In order to fulfill this crucial need, we propose a novel paradigm that we named conformation-based hidden Markov models (COHMMs). This new formalism classifies VO sequences by embedding the nodes of an HMM state transition graph in a Euclidean vector space. This is accomplished by modeling the noise contained in the shape composed by the VO sequence. We cover the one-level as well as the multilevel COHMMs. Five problems are assigned to a multilevel COHMM: 1) sequence probability evaluation, 2) statistical decoding, 3) structural decoding, 4) shape decoding, and 5) learning. We have applied the COHMMs formalism to human face identification tested on different benchmarked face databases. The results show that the multilevel COHMMs outperform the embedded HMMs as well as some standard HMM-based models. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fast Vision Through Frameless Event-Based Sensing and Convolutional Processing: Application to Texture Recognition

    Page(s): 609 - 620
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1968 KB) |  | HTML iconHTML  

    Address-event representation (AER) is an emergent hardware technology which shows a high potential for providing in the near future a solid technological substrate for emulating brain-like processing structures. When used for vision, AER sensors and processors are not restricted to capturing and processing still image frames, as in commercial frame-based video technology, but sense and process visual information in a pixel-level event-based frameless manner. As a result, vision processing is practically simultaneous to vision sensing, since there is no need to wait for sensing full frames. Also, only meaningful information is sensed, communicated, and processed. Of special interest for brain-like vision processing are some already reported AER convolutional chips, which have revealed a very high computational throughput as well as the possibility of assembling large convolutional neural networks in a modular fashion. It is expected that in a near future we may witness the appearance of large scale convolutional neural networks with hundreds or thousands of individual modules. In the meantime, some research is needed to investigate how to assemble and configure such large scale convolutional networks for specific applications. In this paper, we analyze AER spiking convolutional neural networks for texture recognition hardware applications. Based on the performance figures of already available individual AER convolution chips, we emulate large scale networks using a custom made event-based behavioral simulator. We have developed a new event-based processing architecture that emulates with AER hardware Manjunath's frame-based feature recognition software algorithm, and have analyzed its performance using our behavioral simulator. Recognition rate performance is not degraded. However, regarding speed, we show that recognition can be achieved before an equivalent frame is fully sensed and transmitted. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalized Low-Rank Approximations of Matrices Revisited

    Page(s): 621 - 632
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (947 KB) |  | HTML iconHTML  

    Compared to singular value decomposition (SVD), generalized low-rank approximations of matrices (GLRAM) can consume less computation time, obtain higher compression ratio, and yield competitive classification performance. GLRAM has been successfully applied to applications such as image compression and retrieval, and quite a few extensions have been successively proposed. However, in literature, some basic properties and crucial problems with regard to GLRAM have not been explored or solved yet. For this sake, we revisit GLRAM in this paper. First, we reveal such a close relationship between GLRAM and SVD that GLRAM's objective function is identical to SVD's objective function except the imposed constraints. Second, we derive a lower bound of GLRAM's objective function, and discuss when the lower bound can be touched. Moreover, from the viewpoint of minimizing the lower bound, we answer one open problem raised by Ye (Machine Learning, 2005), i.e., a theoretical justification of the experimental phenomenon that, under given number of reduced dimension, the lowest reconstruction error is obtained when the left and right transformations have equal number of columns. Third, we explore when and why GLRAM can perform well in terms of compression, which is a fundamental problem concerning the usability of GLRAM. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Large-Scale Pattern Storage and Retrieval Using Generalized Brain-State-in-a-Box Neural Networks

    Page(s): 633 - 643
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2449 KB) |  | HTML iconHTML  

    In this paper, a generalized Brain-State-in-a-Box (gBSB)-based hybrid neural network is proposed for storing and retrieving pattern sequences. The hybrid network consists of autoassociative and heteroassociative parts. Then, a large-scale image storage and retrieval neural system is constructed using the gBSB-based hybrid neural network and the pattern decomposition concept. The notion of the deadbeat stability is employed to describe the stability property of the vertices of the hypercube to which the trajectories of the gBSB neural system are constrained. Extensive simulations of large scale pattern and image storing and retrieval are presented to illustrate the results obtained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Simplifying Mixture Models Through Function Approximation

    Page(s): 644 - 658
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (864 KB) |  | HTML iconHTML  

    The finite mixture model is widely used in various statistical learning problems. However, the model obtained may contain a large number of components, making it inefficient in practical applications. In this paper, we propose to simplify the mixture model by minimizing an upper bound of the approximation error between the original and the simplified model, under the use of the L 2 distance measure. This is achieved by first grouping similar components together and then performing local fitting through function approximation. The simplified model obtained can then be used as a replacement of the original model to speed up various algorithms involving mixture models during training (e.g., Bayesian filtering, belief propagation) and testing [e.g., kernel density estimation, support vector machine (SVM) testing]. Encouraging results are observed in the experiments on density estimation, clustering-based image segmentation, and simplification of SVM decision functions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Boosting Through Optimization of Margin Distributions

    Page(s): 659 - 666
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (577 KB) |  | HTML iconHTML  

    Boosting has been of great interest recently in the machine learning community because of the impressive performance for classification and regression problems. The success of boosting algorithms may be interpreted in terms of the margin theory. Recently, it has been shown that generalization error of classifiers can be obtained by explicitly taking the margin distribution of the training data into account. Most of the current boosting algorithms in practice usually optimize a convex loss function and do not make use of the margin distribution. In this brief, we design a new boosting algorithm, termed margin-distribution boosting (MDBoost), which directly maximizes the average margin and minimizes the margin variance at the same time. This way the margin distribution is optimized. A totally corrective optimization algorithm based on column generation is proposed to implement MDBoost. Experiments on various data sets show that MDBoost outperforms AdaBoost and LPBoost in most cases. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Conic Section Function Neural Network Circuitry for Offline Signature Recognition

    Page(s): 667 - 672
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (317 KB) |  | HTML iconHTML  

    In this brief, conic section function neural network (CSFNN) circuitry was designed for offline signature recognition. CSFNN is a unified framework for multilayer perceptron (MLP) and radial basis function (RBF) networks to make simultaneous use of advantages of both. The CSFNN circuitry architecture was developed using a mixed mode circuit implementation. The designed circuit system is problem independent. Hence, the general purpose neural network circuit system could be applied to various pattern recognition problems with different network sizes on condition with the maximum network size of 16-16-8. In this brief, CSFNN circuitry system has been applied to two different signature recognition problems. CSFNN circuitry was trained with chip-in-the-loop learning technique in order to compensate typical analog process variations. CSFNN hardware achieved highly comparable computational performances with CSFNN software for nonlinear signature recognition problems. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Black-Box Identification of a Class of Nonlinear Systems by a Recurrent Neurofuzzy Network

    Page(s): 672 - 679
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1346 KB) |  | HTML iconHTML  

    This brief presents a structure for black-box identification based on continuous-time recurrent neurofuzzy networks for a class of dynamic nonlinear systems. The proposed network catches the dynamics of a system by generating its own states, using only input and output measurements of the system. The training algorithm is based on adaptive observer theory, the stability of the network, the convergence of the training algorithm, and the ultimate bound on the identification error as well as the parameter error are established. Experimental results are included to illustrate the effectiveness of the proposed method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Memory-Efficient Fully Coupled Filtering Approach for Observational Model Building

    Page(s): 680 - 686
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (593 KB) |  | HTML iconHTML  

    Generally, training neural networks with the global extended Kalman filter (GEKF) technique exhibits excellent performance at the expense of a large increase in computational costs which can become prohibitive even for networks of moderate size. This drawback was previously addressed by heuristically decoupling some of the weights of the networks. Inevitably, ad hoc decoupling leads to a degradation in the quality (accuracy) of the resultant neural networks. In this paper, we present an algorithm that emulates the accuracy of GEKF, but avoids the construction of the state covariance matrix-the source of the computational bottleneck in GEKF. In the proposed algorithm, all the synaptic weights remain connected while the amount of computer memory required is similar to (or cheaper than) the memory requirements in the decoupling schemes. We also point out that the new method can be extended to derivative-free nonlinear Kalman filters, such as the unscented Kalman filter and ensemble Kalman filters. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Lattice Point Sets for Deterministic Learning and Approximate Optimization Problems

    Page(s): 687 - 692
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (285 KB) |  | HTML iconHTML  

    In this brief, the use of lattice point sets (LPSs) is investigated in the context of general learning problems (including function estimation and dynamic optimization), in the case where the classic empirical risk minimization (ERM) principle is considered and there is freedom to choose the sampling points of the input space. Here it is proved that convergence of the ERM principle is guaranteed when LPSs are employed as training sets for the learning procedure, yielding up to a superlinear convergence rate under some regularity hypotheses on the involved functions. Preliminary simulation results are also provided. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improved Delay-Dependent Stability Condition of Discrete Recurrent Neural Networks With Time-Varying Delays

    Page(s): 692 - 697
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (210 KB) |  | HTML iconHTML  

    This brief investigates the problem of global exponential stability analysis for discrete recurrent neural networks with time-varying delays. In terms of linear matrix inequality (LMI) approach, a novel delay-dependent stability criterion is established for the considered recurrent neural networks via a new Lyapunov function. The obtained condition has less conservativeness and less number of variables than the existing ones. Numerical example is given to demonstrate the effectiveness of the proposed method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Special issue on white box nonlinear prediction models

    Page(s): 698
    Save to Project icon | Request Permissions | PDF file iconPDF (151 KB)  
    Freely Available from IEEE
  • 2010 IEEE World Congress on Computational Intelligence

    Page(s): 699
    Save to Project icon | Request Permissions | PDF file iconPDF (2434 KB)  
    Freely Available from IEEE
  • Explore IEL IEEE's most comprehensive resource [advertisement]

    Page(s): 700
    Save to Project icon | Request Permissions | PDF file iconPDF (345 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Page(s): C3
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks Information for authors

    Page(s): C4
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope