By Topic

Neural Networks, IEEE Transactions on

Issue 4 • Date July 2006

Filter Results

Displaying Results 1 - 25 of 30
  • Table of contents

    Page(s): c1 - c4
    Save to Project icon | Request Permissions | PDF file iconPDF (40 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): c2
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • A statistical property of multiagent learning based on Markov decision process

    Page(s): 829 - 842
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (664 KB) |  | HTML iconHTML  

    We exhibit an important property called the asymptotic equipartition property (AEP) on empirical sequences in an ergodic multiagent Markov decision process (MDP). Using the AEP which facilitates the analysis of multiagent learning, we give a statistical property of multiagent learning, such as reinforcement learning (RL), near the end of the learning process. We examine the effect of the conditions among the agents on the achievement of a cooperative policy in three different cases: blind, visible, and communicable. Also, we derive a bound on the speed with which the empirical sequence converges to the best sequence in probability, so that the multiagent learning yields the best cooperative result. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning lateral interactions for feature binding and sensory segmentation from prototypic basis interactions

    Page(s): 843 - 862
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2232 KB) |  | HTML iconHTML  

    We present a hybrid learning method bridging the fields of recurrent neural networks, unsupervised Hebbian learning, vector quantization, and supervised learning to implement a sophisticated image and feature segmentation architecture. This architecture is based on the competitive layer model (CLM), a dynamic feature binding model, which is applicable on a wide range of perceptual grouping and segmentation problems. A predefined target segmentation can be achieved as attractor states of this linear threshold recurrent network, if the lateral weights are chosen by Hebbian learning. The weight matrix is given by the correlation matrix of special pattern vectors with a structure dependent on the target labeling. Generalization is achieved by applying vector quantization on pair-wise feature relations, like proximity and similarity, defined by external knowledge. We show the successful application of the method to a number of artificial test examples and a medical image segmentation problem of fluorescence microscope cell images. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Real-time learning capability of neural networks

    Page(s): 863 - 878
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1440 KB) |  | HTML iconHTML  

    In some practical applications of neural networks, fast response to external events within an extremely short time is highly demanded and expected. However, the extensively used gradient-descent-based learning algorithms obviously cannot satisfy the real-time learning needs in many applications, especially for large-scale applications and/or when higher generalization performance is required. Based on Huang's constructive network model, this paper proposes a simple learning algorithm capable of real-time learning which can automatically select appropriate values of neural quantizers and analytically determine the parameters (weights and bias) of the network at one time only. The performance of the proposed algorithm has been systematically investigated on a large batch of benchmark real-world regression and classification problems. The experimental results demonstrate that our algorithm can not only produce good generalization performance but also have real-time learning and prediction capability. Thus, it may provide an alternative approach for the practical applications of neural networks where real-time learning and prediction implementation is required. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal approximation using incremental constructive feedforward networks with random hidden nodes

    Page(s): 879 - 892
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (952 KB) |  | HTML iconHTML  

    According to conventional neural network theories, single-hidden-layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implementations, tuning all the parameters of the networks may cause learning complicated and inefficient, and it may be difficult to train networks with nondifferential activation functions such as threshold networks. Unlike conventional neural network theories, this paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer. In such SLFNs implementations, the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g:R→R and the activation functions for RBF nodes can be any integrable piecewise continuous functions g:R→R and ∫Rg(x)dx≠0. The proposed incremental method is efficient not only for SFLNs with continuous (including nondifferentiable) activation functions but also for SLFNs with piecewise continuous (such as threshold) activation functions. Compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A study on SMO-type decomposition methods for support vector machines

    Page(s): 893 - 908
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (696 KB) |  | HTML iconHTML  

    Decomposition methods are currently one of the major methods for training support vector machines. They vary mainly according to different working set selections. Existing implementations and analysis usually consider some specific selection rules. This paper studies sequential minimal optimization type decomposition methods under a general and flexible way of choosing the two-element working set. The main results include: 1) a simple asymptotic convergence proof, 2) a general explanation of the shrinking and caching techniques, and 3) the linear convergence of the methods. Extensions to some support vector machine variants are also discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cooperative information maximization with Gaussian activation functions for self-organizing maps

    Page(s): 909 - 918
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (600 KB) |  | HTML iconHTML  

    In this paper, we propose a new information-theoretic method to produce explicit self-organizing maps (SOMs). Competition is realized by maximizing mutual information between input patterns and competitive units. Competitive unit outputs are computed by the Gaussian function of distance between input patterns and competitive units. A property of this Gaussian function is that, as distance becomes smaller, a neuron tends to fire strongly. Cooperation processes are realized by taking into account the firing rates of neighboring neurons. We applied our method to uniform distribution learning, chemical compound classification and road classification. Experimental results confirmed that cooperation processes could significantly increase information content in input patterns. When cooperative operations are not effective in increasing information, mutual information as well as entropy maximization is used to increase information. Experimental results showed that entropy maximization could be used to increase information and to obtain clearer SOMs, because competitive units are forced to be equally used on average. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Motif discoveries in unaligned molecular sequences using self-organizing neural networks

    Page(s): 919 - 928
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB) |  | HTML iconHTML  

    In this paper, we study the problem of motif discoveries in unaligned DNA and protein sequences. The problem of motif identification in DNA and protein sequences has been studied for many years in the literature. Major hurdles at this point include computational complexity and reliability of the search algorithms. We propose a self-organizing neural network structure for solving the problem of motif identification in DNA and protein sequences. Our network contains several layers, with each layer performing classifications at different levels. The top layer divides the input space into a small number of regions and the bottom layer classifies all input patterns into motifs and nonmotif patterns. Depending on the number of input patterns to be classified, several layers between the top layer and the bottom layer are needed to perform intermediate classifications. We maintain a low computational complexity through the use of the layered structure so that each pattern's classification is performed with respect to a small subspace of the whole input space. Our self-organizing neural network will grow as needed (e.g., when more motif patterns are classified). It will give the same amount of attention to each input pattern and will not omit any potential motif patterns. Finally, simulation results show that our algorithm outperforms existing algorithms in certain aspects. In particular, simulation results show that our algorithm can identify motifs with more mutations than existing algorithms. Our algorithm works well for long DNA sequences as well. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fuzzy ARTMAP with input relevances

    Page(s): 929 - 941
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (608 KB) |  | HTML iconHTML  

    We introduce a new fuzzy ARTMAP (FAM) neural network: Fuzzy ARTMAP with relevance factor (FAMR). The FAMR architecture is able to incrementally "grow" and to sequentially accommodate input-output sample pairs. Each training pair has a relevance factor assigned to it, proportional to the importance of that pair during the learning phase. The relevance factors are user-defined or computed. The FAMR can be trained as a classifier and, at the same time, as a nonparametric estimator of the probability that an input belongs to a given class. The FAMR probability estimation converges almost surely and in the mean square to the posterior probability. Our theoretical results also characterize the convergence rate of the approximation. Using a relevance factor adds more flexibility to the training phase, allowing ranking of sample pairs according to the confidence we have in the information source. We analyze the FAMR capability for mapping noisy functions when training data originates from multiple sources with known levels of noise. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On global-local artificial neural networks for function approximation

    Page(s): 942 - 952
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (616 KB) |  | HTML iconHTML  

    We present a hybrid radial basis function (RBF) sigmoid neural network with a three-step training algorithm that utilizes both global search and gradient descent training. The algorithm used is intended to identify global features of an input-output relationship before adding local detail to the approximating function. It aims to achieve efficient function approximation through the separate identification of aspects of a relationship that are expressed universally from those that vary only within particular regions of the input space. We test the effectiveness of our method using five regression tasks; four use synthetic datasets while the last problem uses real-world data on the wave overtopping of seawalls. It is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways: lower mean square errors are often achievable using fewer hidden neurons and with less need for regularization. Our global-local artificial neural network (GL-ANN) is also seen to compare favorably with both perceptron radial basis net and regression tree derived RBFs. A number of issues concerning the training of GL-ANNs are discussed: the use of regularization, the inclusion of a gradient descent optimization step, the choice of RBF spreads, model selection, and the development of appropriate stopping criteria. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stable neurovisual servoing for robot manipulators

    Page(s): 953 - 965
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1424 KB) |  | HTML iconHTML  

    In this paper, we propose a stable neurovisual servoing algorithm for set-point control of planar robot manipulators in a fixed-camera configuration an show that all the closed-loop signals are uniformly ultimately bounded (UUB) and converge exponentially to a small compact set. We assume that the gravity term and Jacobian matrix are unknown. Radial basis function neural networks (RBFNNs) with online real-time learning are proposed for compensating both gravitational forces and errors in the robot Jacobian matrix. The learning rule for updating the neural network weights, similar to a back propagation algorithm, is obtained from a Lyapunov stability analysis. Experimental results on a two degrees of freedom manipulator are presented to evaluate the proposed controller. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An incremental training method for the probabilistic RBF network

    Page(s): 966 - 974
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (792 KB) |  | HTML iconHTML  

    The probabilistic radial basis function (PRBF) network constitutes a probabilistic version of the RBF network for classification that extends the typical mixture model approach to classification by allowing the sharing of mixture components among all classes. The typical learning method of PRBF for a classification task employs the expectation-maximization (EM) algorithm and depends strongly on the initial parameter values. In this paper, we propose a technique for incremental training of the PRBF network for classification. The proposed algorithm starts with a single component and incrementally adds more components at appropriate positions in the data space. The addition of a new component is based on criteria for detecting a region in the data space that is crucial for the classification task. After the addition of all components, the algorithm splits every component of the network into subcomponents, each one corresponding to a different class. Experimental results using several well-known classification data sets indicate that the incremental method provides solutions of superior classification performance compared to the hierarchical PRBF training method. We also conducted comparative experiments with the support vector machines method and present the obtained results along with a qualitative comparison of the two approaches. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonlinear spatial-temporal prediction based on optimal fusion

    Page(s): 975 - 988
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2376 KB) |  | HTML iconHTML  

    The problem of spatial-temporal signal processing and modeling has been of great interest in recent years. A new spatial-temporal prediction method is presented in this paper. An optimal fusion scheme based on fourth-order statistic is first employed to combine the received signals at different spatial domains. The fused signal is then used to construct a spatial-temporal predictor by a support vector machine. It is shown theoretically that the proposed method has an improved performance even in non-Gaussian environments. To demonstrate the practicality of this spatial-temporal predictor, we apply it to model real-life radar sea scattered signals. Experimental results show that the proposed method can provide a more accurate model for sea clutter than the conventional methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A gradual noisy chaotic neural network for solving the broadcast scheduling problem in packet radio networks

    Page(s): 989 - 1000
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (664 KB) |  | HTML iconHTML  

    In this paper, we propose a gradual noisy chaotic neural network (G-NCNN) to solve the NP-complete broadcast scheduling problem (BSP) in packet radio networks. The objective of the BSP is to design an optimal time-division multiple-access (TDMA) frame structure with minimal TDMA frame length and maximal channel utilization. A two-phase optimization is adopted to achieve the two objectives with two different energy functions, so that the G-NCNN not only finds the minimum TDMA frame length but also maximizes the total node transmissions. In the first phase, we propose a G-NCNN which combines the noisy chaotic neural network (NCNN) and the gradual expansion scheme to find a minimal TDMA frame length. In the second phase, the NCNN is used to find maximal node transmissions in the TDMA frame obtained in the first phase. The performance is evaluated through several benchmark examples and 600 randomly generated instances. The results show that the G-NCNN outperforms previous approaches, such as mean field annealing, a hybrid Hopfield network-genetic algorithm, the sequential vertex coloring algorithm, and the gradual neural network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Polymer property prediction and optimization using neural networks

    Page(s): 1001 - 1014
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (744 KB) |  | HTML iconHTML  

    Prediction and optimization of polymer properties is a complex and highly nonlinear problem with no easy method to predict polymer properties directly and accurately. The problem is especially complicated with high molecular weight polymers such as engineering plastics which have the greatest use in industry. The effect of modifying a monomer (polymer repeat unit) on polymerization and the resulting polymer properties is not easy to investigate experimentally given the large number of possible changes. This severely curtails the design of new polymers with specific end-use properties. In this paper, we show how properties of modified monomers can be predicted using neural networks. We utilize a database of polymer properties and employ a variety of networks ranging from backpropagation networks to unsupervised self-associating maps. We select particular networks that accurately predict specific polymer properties. These networks are classified into groups that range from those that provide quick training to those that provide excellent generalization. We also show how the available polymer database can be used to accurately predict and optimize polymer properties. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Processing of chemical sensor arrays with a biologically inspired model of olfactory coding

    Page(s): 1015 - 1024
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1312 KB) |  | HTML iconHTML  

    This paper presents a computational model for chemical sensor arrays inspired by the first two stages in the olfactory pathway: distributed coding with olfactory receptor neurons and chemotopic convergence onto glomerular units. We propose a monotonic concentration-response model that maps conventional sensor-array inputs into a distributed activation pattern across a large population of neuroreceptors. Projection onto glomerular units in the olfactory bulb is then simulated with a self-organizing model of chemotopic convergence. The pattern recognition performance of the model is characterized using a database of odor patterns from an array of temperature modulated chemical sensors. The chemotopic code achieved by the proposed model is shown to improve the signal-to-noise ratio available at the sensor inputs while being consistent with results from neurobiology. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Design and implementation of multipattern generators in analog VLSI

    Page(s): 1025 - 1038
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2112 KB) |  | HTML iconHTML  

    In recent years, computational biologists have shown through simulation that small neural networks with fixed connectivity are capable of producing multiple output rhythms in response to transient inputs. It is believed that such networks may play a key role in certain biological behaviors such as dynamic gait control. In this paper, we present a novel method for designing continuous-time recurrent neural networks (CTRNNs) that contain multiple embedded limit cycles, and we show that it is possible to switch the networks between these embedded limit cycles with simple transient inputs. We also describe the design and testing of a fully integrated four-neuron CTRNN chip that is used to implement the neural network pattern generators. We provide two example multipattern generators and show that the measured waveforms from the chip agree well with numerical simulations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Parallel sequential minimal optimization for the training of support vector machines

    Page(s): 1039 - 1049
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (592 KB)  

    Sequential minimal optimization (SMO) is one popular algorithm for training support vector machine (SVM), but it still requires a large amount of computation time for solving large size problems. This paper proposes one parallel implementation of SMO for training SVM. The parallel SMO is developed using message passing interface (MPI). Specifically, the parallel SMO first partitions the entire training data set into smaller subsets and then simultaneously runs multiple CPU processors to deal with each of the partitioned data sets. Experiments show that there is great speedup on the adult data set and the Mixing National Institute of Standard and Technology (MNIST) data set when many processors are used. There are also satisfactory results on the Web data set. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Real-time computing platform for spiking neurons (RT-spike)

    Page(s): 1050 - 1063
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1232 KB) |  | HTML iconHTML  

    A computing platform is described for simulating arbitrary networks of spiking neurons in real time. A hybrid computing scheme is adopted that uses both software and hardware components to manage the tradeoff between flexibility and computational power; the neuron model is implemented in hardware and the network model and the learning are implemented in software. The incremental transition of the software components into hardware is supported. We focus on a spike response model (SRM) for a neuron where the synapses are modeled as input-driven conductances. The temporal dynamics of the synaptic integration process are modeled with a synaptic time constant that results in a gradual injection of charge. This type of model is computationally expensive and is not easily amenable to existing software-based event-driven approaches. As an alternative we have designed an efficient time-based computing architecture in hardware, where the different stages of the neuron model are processed in parallel. Further improvements occur by computing multiple neurons in parallel using multiple processing units. This design is tested using reconfigurable hardware and its scalability and performance evaluated. Our overall goal is to investigate biologically realistic models for the real-time control of robots operating within closed action-perception loops, and so we evaluate the performance of the system on simulating a model of the cerebellum where the emulation of the temporal dynamics of the synaptic integration process is important. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A fast identification algorithm for box-cox transformation based radial basis function neural network

    Page(s): 1064 - 1069
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB) |  | HTML iconHTML  

    In this letter, a Box-Cox transformation-based radial basis function (RBF) neural network is introduced using the RBF neural network to represent the transformed system output. Initially a fixed and moderate sized RBF model base is derived based on a rank revealing orthogonal matrix triangularization (QR decomposition). Then a new fast identification algorithm is introduced using Gauss-Newton algorithm to derive the required Box-Cox transformation, based on a maximum likelihood estimator. The main contribution of this letter is to explore the special structure of the proposed RBF neural network for computational efficiency by utilizing the inverse of matrix block decomposition lemma. Finally, the Box-Cox transformation-based RBF neural network, with good generalization and sparsity, is identified based on the derived optimal Box-Cox transformation and a D-optimality-based orthogonal forward regression algorithm. The proposed algorithm and its efficacy are demonstrated with an illustrative example in comparison with support vector machine regression. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural network mechanism for the orientation behavior of sand scorpions towards prey

    Page(s): 1070 - 1076
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (560 KB) |  | HTML iconHTML  

    Sand scorpions use their tactile sense organs on their legs to capture their prey. They are able to localize their prey by processing vibration signals generated by the prey movement. The central nervous system receives stimulus-locked neuron firings of the sense organs on their eight legs. It is believed that eight receptor neurons in the brain interact with each other with triad inhibitions and then a voting contribution of the receptor neurons is calculated to obtain the resource direction. This letter presents a neuronal model of the voting procedure to locate prey. The neural network consists of a sinusoidal array of neurons for the resource vector, and it has been tested on the orientation data of scorpions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Delay-dependent state estimation for delayed neural networks

    Page(s): 1077 - 1081
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (200 KB) |  | HTML iconHTML  

    In this letter, the delay-dependent state estimation problem for neural networks with time-varying delay is investigated. A delay-dependent criterion is established to estimate the neuron states through available output measurements such that the dynamics of the estimation error is globally exponentially stable. The proposed method is based on the free-weighting matrix approach and is applicable to the case that the derivative of a time-varying delay takes any value. An algorithm is presented to compute the state estimator. Finally, a numerical example is given to demonstrate the effectiveness of this approach and the improvement over existing ones. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Face recognition using kernel scatter-difference-based discriminant analysis

    Page(s): 1081 - 1085
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (344 KB) |  | HTML iconHTML  

    There are two fundamental problems with the Fisher linear discriminant analysis for face recognition. One is the singularity problem of the within-class scatter matrix due to small training sample size. The other is that it cannot efficiently describe complex nonlinear variations of face images because of its linear property. In this letter, a kernel scatter-difference-based discriminant analysis is proposed to overcome these two problems. We first use the nonlinear kernel trick to map the input data into an implicit feature space F. Then a scatter-difference-based discriminant rule is defined to analyze the data in F. The proposed method can not only produce nonlinear discriminant features but also avoid the singularity problem of the within-class scatter matrix. Extensive experiments show encouraging recognition performance of the new algorithm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analog neural network for support vector machine learning

    Page(s): 1085 - 1091
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (350 KB) |  | HTML iconHTML  

    An analog neural network for support vector machine learning is proposed, based on a partially dual formulation of the quadratic programming problem. It results in a simpler circuit implementation with respect to existing neural solutions for the same application. The effectiveness of the proposed network is shown through some computer simulations concerning benchmark problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope