By Topic

Neural Networks, IEEE Transactions on

Issue 2 • Date March 2006

Filter Results

Displaying Results 1 - 25 of 34
  • Table of contents

    Page(s): c1 - c4
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): c2
    Save to Project icon | Request Permissions | PDF file iconPDF (36 KB)  
    Freely Available from IEEE
  • A node pruning algorithm based on a Fourier amplitude sensitivity test method

    Page(s): 273 - 293
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2195 KB)  

    In this paper, we propose a new pruning algorithm to obtain the optimal number of hidden units of a single layer of a fully connected neural network (NN). The technique relies on a global sensitivity analysis of model output. The relevance of the hidden nodes is determined by analysing the Fourier decomposition of the variance of the model output. Each hidden unit is assigned a ratio (the fraction of variance which the unit accounts for) that gives their ranking. This quantitative information therefore leads to a suggestion of the most favorable units to eliminate. Experimental results suggest that the method can be seen as an effective tool available to the user in controlling the complexity in NNs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalizing self-organizing map for categorical data

    Page(s): 294 - 304
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (574 KB)  

    The self-organizing map (SOM) is an unsupervised neural network which projects high-dimensional data onto a low-dimensional grid and visually reveals the topological order of the original data. Self-organizing maps have been successfully applied to many fields, including engineering and business domains. However, the conventional SOM training algorithm handles only numeric data. Categorical data are usually converted to a set of binary data before training of an SOM takes place. If a simple transformation scheme is adopted, the similarity information embedded between categorical values may be lost. Consequently, the trained SOM is unable to reflect the correct topological order. This paper proposes a generalized self-organizing map model that offers an intuitive method of specifying the similarity between categorical values via distance hierarchies and, hence, enables the direct process of categorical values during training. In fact, distance hierarchy unifies the distance computation of both numeric and categorical values. The unification is done by mapping the values to distance hierarchies and then measuring the distance in the hierarchies. Experiments on synthetic and real datasets were conducted, and the results demonstrated the effectiveness of the generalized SOM model. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The parameterless self-organizing map algorithm

    Page(s): 305 - 316
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (832 KB)  

    The parameterless self-organizing map (PLSOM) is a new neural network algorithm based on the self-organizing map (SOM). It eliminates the need for a learning rate and annealing schemes for learning rate and neighborhood size. We discuss the relative performance of the PLSOM and the SOM and demonstrate some tasks in which the SOM fails but the PLSOM performs satisfactory. Finally we discuss some example applications of the PLSOM and present a proof of ordering under certain limited conditions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Implementing online natural gradient learning: problems and solutions

    Page(s): 317 - 329
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (916 KB)  

    The online natural gradient learning is an efficient algorithm to resolve the slow learning speed and poor performance of the standard gradient descent method. However, there are several problems to implement this algorithm. In this paper, we proposed a new algorithm to solve these problems and then compared the new algorithm with other known algorithms for online learning, including Almeida-Langlois-Amaral-Plakhov algorithm (ALAP), Vario-η, local adaptive learning rate and learning with momentum etc., using sample data sets from Proben1 and normalized handwritten digits, automatically scanned from envelopes by the U.S. Postal Services. The strong and weak points of these algorithms were analyzed and tested empirically. We found out that using the online training error as the criterion to determine whether the learning rate should be changed or not is not appropriate and our new algorithm has better performance than other existing online algorithms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The linear separability problem: some testing methods

    Page(s): 330 - 344
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (632 KB)  

    The notion of linear separability is used widely in machine learning research. Learning algorithms that use this concept to learn include neural networks (single layer perceptron and recursive deterministic perceptron), and kernel machines (support vector machines). This paper presents an overview of several of the methods for testing linear separability between two classes. The methods are divided into four groups: Those based on linear programming, those based on computational geometry, one based on neural networks, and one based on quadratic programming. The Fisher linear discriminant method is also presented. A section on the quantification of the complexity of classification problems is included. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Modulated Hebb-Oja learning Rule-a method for principal subspace analysis

    Page(s): 345 - 356
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (523 KB)  

    This paper presents analysis of the recently proposed modulated Hebb-Oja (MHO) method that performs linear mapping to a lower-dimensional subspace. Principal component subspace is the method that will be analyzed. Comparing to some other well-known methods for yielding principal component subspace (e.g., Oja's Subspace Learning Algorithm), the proposed method has one feature that could be seen as desirable from the biological point of view-synaptic efficacy learning rule does not need the explicit information about the value of the other efficacies to make individual efficacy modification. Also, the simplicity of the "neural circuits" that perform global computations and a fact that their number does not depend on the number of input and output neurons, could be seen as good features of the proposed method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Toward the training of feed-forward neural networks with the D-optimum input sequence

    Page(s): 357 - 373
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1135 KB)  

    The problem under consideration is to obtain a measurement schedule for training neural networks. This task is perceived as an experimental design in a given design space that is obtained in such a way as to minimize the difference between the neural network and the system being considered. This difference can be expressed in many different ways and one of them, namely, the D-optimality criterion is used in this paper. In particular, the paper presents a unified and comprehensive treatment of this problem by discussing the existing and previously unpublished properties of the optimum experimental design (OED) for neural networks. The consequences of the above properties are discussed as well. A hybrid algorithm that can be used for both the training and data development of neural networks is another important contribution of this paper. A careful analysis of the algorithm is presented and its comprehensive convergence analysis with the help of the Lyapunov method are given. The paper contains a number of numerical examples that justify the application of the OED theory for neural networks. Moreover, an industrial application example is given that deals with the valve actuator. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach

    Page(s): 374 - 384
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (432 KB)  

    There is much interest in rule extraction from neural networks and a plethora of different methods have been proposed for this purpose. We discuss the merits of pedagogical and decompositional approaches to rule extraction from trained neural networks, and show that some currently used methods for binary data comply with a theoretical formalism for extraction of Boolean rules from continuously valued logic. This formalism is extended into a generic methodology for rule extraction from smooth decision surfaces fitted to discrete or quantized continuous variables independently of the analytical structure of the underlying model, and in a manner that is efficient even for high input dimensions. This methodology is then tested with Monks' data, for which exact rules are obtained and to Wisconsin's breast cancer data, where a small number of high-order rules are identified whose discriminatory performance can be directly visualized. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A bidirectional heteroassociative memory for binary and grey-level patterns

    Page(s): 385 - 396
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1318 KB)  

    Typical bidirectional associative memories (BAM) use an offline, one-shot learning rule, have poor memory storage capacity, are sensitive to noise, and are subject to spurious steady states during recall. Recent work on BAM has improved network performance in relation to noisy recall and the number of spurious attractors, but at the cost of an increase in BAM complexity. In all cases, the networks can only recall bipolar stimuli and, thus, are of limited use for grey-level pattern recall. In this paper, we introduce a new bidirectional heteroassociative memory model that uses a simple self-convergent iterative learning rule and a new nonlinear output function. As a result, the model can learn online without being subject to overlearning. Our simulation results show that this new model causes fewer spurious attractors when compared to others popular BAM networks, for a comparable performance in terms of tolerance to noise and storage capacity. In addition, the novel output function enables it to learn and recall grey-level patterns in a bidirectional way. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Basins of attraction in fully asynchronous discrete-time discrete-state dynamic networks

    Page(s): 397 - 408
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (408 KB)  

    This paper gives a formulation of the basins of fixed point states of fully asynchronous discrete-time discrete-state dynamic networks. That formulation provides two advantages. The first one is to point out the different behaviors between synchronous and asynchronous modes and the second one is to allow us to easily deduce an algorithm which determines the behavior of a network for a given initialization. In the context of this study, we consider networks of a large number of neurons (or units, processors, etc.), whose dynamic is fully asynchronous with overlapping updates . We suppose that the neurons take a finite number of discrete states and that the updating scheme is discrete in time. We make no hypothesis on the activation functions of the nodes, so that the dynamic of the network may have multiple cycles and/or basins. Our results are illustrated on a simple example of a fully asynchronous Hopfield neural network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Dynamics analysis and analog associative memory of networks with LT neurons

    Page(s): 409 - 418
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (822 KB)  

    The additive recurrent network structure of linear threshold neurons represents a class of biologically-motivated models, where nonsaturating transfer functions are necessary for representing neuronal activities, such as that of cortical neurons. This paper extends the existing results of dynamics analysis of such linear threshold networks by establishing new and milder conditions for boundedness and asymptotical stability, while allowing for multistability. As a condition for asymptotical stability, it is found that boundedness does not require a deterministic matrix to be symmetric or possess positive off-diagonal entries. The conditions put forward an explicit way to design and analyze such networks. Based on the established theory, an alternate approach to study such networks is through permitted and forbidden sets. An application of the linear threshold (LT) network is analog associative memory, for which a simple design method describing the associative memory is suggested in this paper. The proposed design method is similar to a generalized Hebbian approach, but with distinctions of additional network parameters for normalization, excitation and inhibition, both on a global and local scale. The computational abilities of the network are dependent on its nonlinear dynamics, which in turn is reliant upon the sparsity of the memory vectors. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Blind estimation of channel parameters and source components for EEG signals: a sparse factorization approach

    Page(s): 419 - 431
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (598 KB)  

    In this paper, we use a two-stage sparse factorization approach for blindly estimating the channel parameters and then estimating source components for electroencephalogram (EEG) signals. EEG signals are assumed to be linear mixtures of source components, artifacts, etc. Therefore, a raw EEG data matrix can be factored into the product of two matrices, one of which represents the mixing matrix and the other the source component matrix. Furthermore, the components are sparse in the time-frequency domain, i.e., the factorization is a sparse factorization in the time frequency domain. It is a challenging task to estimate the mixing matrix. Our extensive analysis and computational results, which were based on many sets of EEG data, not only provide firm evidences supporting the above assumption, but also prompt us to propose a new algorithm for estimating the mixing matrix. After the mixing matrix is estimated, the source components are estimated in the time frequency domain using a linear programming method. In an example of the potential applications of our approach, we analyzed the EEG data that was obtained from a modified Sternberg memory experiment. Two almost uncorrelated components obtained by applying the sparse factorization method were selected for phase synchronization analysis. Several interesting findings were obtained, especially that memory-related synchronization and desynchronization appear in the alpha band, and that the strength of alpha band synchronization is related to memory performance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive wavelet neural network control with hysteresis estimation for piezo-positioning mechanism

    Page(s): 432 - 444
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (778 KB)  

    An adaptive wavelet neural network (AWNN) control with hysteresis estimation is proposed in this study to improve the control performance of a piezo-positioning mechanism, which is always severely deteriorated due to hysteresis effect. First, the control system configuration of the piezo-positioning mechanism is introduced. Then, a new hysteretic model by integrating a modified hysteresis friction force function is proposed to represent the dynamics of the overall piezo-positioning mechanism. According to this developed dynamics, an AWNN controller with hysteresis estimation is proposed. In the proposed AWNN controller, a wavelet neural network (WNN) with accurate approximation capability is employed to approximate the part of the unknown function in the proposed dynamics of the piezo-positioning mechanism, and a robust compensator is proposed to confront the lumped uncertainty that comprises the inevitable approximation errors due to finite number of wavelet basis functions and disturbances, optimal parameter vectors, and higher order terms in Taylor series. Moreover, adaptive learning algorithms for the online learning of the parameters of the WNN are derived based on the Lyapunov stability theorem. Finally, the command tracking performance and the robustness to external load disturbance of the proposed AWNN control system are illustrated by some experimental results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Modeling and inverse controller design for an unmanned aerial vehicle based on the self-organizing map

    Page(s): 445 - 460
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1243 KB)  

    The next generation of aircraft will have dynamics that vary considerably over the operating regime. A single controller will have difficulty to meet the design specifications. In this paper, a self-organizing map (SOM)-based local linear modeling scheme of an unmanned aerial vehicle (UAV) is developed to design a set of inverse controllers. The SOM selects the operating regime depending only on the embedded output space information and avoids normalization of the input data. Each local linear model is associated with a linear controller, which is easy to design. Switching of the controllers is done synchronously with the active local linear model that tracks the different operating conditions. The proposed multiple modeling and control strategy has been successfully tested in a simulator that models the LoFLYTE UAV. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural-network-based adaptive UPFC for improving transient stability performance of power system

    Page(s): 461 - 470
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (445 KB)  

    This paper uses the recently proposed H-learning method, for updating the parameter of the radial basis function neural network (RBFNN) used as a control scheme for the unified power flow controller (UPFC) to improve the transient stability performance of a multimachine power system. The RBFNN uses a single neuron architecture whose input is proportional to the difference in error and the updating of its parameters is carried via a proportional value of the error. Also, the coefficients of the difference of error, error, and auxiliary signal used for improving damping performance are depicted by a genetic algorithm. The performance of the newly designed controller is evaluated in a four-machine power system subjected to different types of disturbances. The newly designed single-neuron RBFNN-based UPFC exhibits better damping performance compared to the conventional PID as well as the extended Kalman filter (EKF) updating-based RBFNN scheme, making the unstable cases stable. Its simple architecture reduces the computational burden, thereby making it attractive for real-time implementation. Also, all the machines are being equipped with the conventional power system stabilizer (PSS) to study the coordinated effect of UPFC and PSS in the system. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sparse bayesian kernel survival analysis for modeling the growth domain of microbial pathogens

    Page(s): 471 - 481
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (681 KB)  

    Survival analysis is a branch of statistics concerned with the time elapsing before "failure," with diverse applications in medical statistics and the analysis of the reliability of electrical or mechanical components. We introduce a parametric accelerated life survival analysis model based on kernel learning methods that, at least in principal, is able to learn arbitrary dependencies between a vector of explanatory variables and the scale of the distribution of survival times. The proposed kernel survival analysis method is then used to model the growth domain of Clostridium botulinum, the food processing and storage conditions permitting the growth of this foodborne microbial pathogen, leading to the production of the neurotoxin responsible for botulism. A Bayesian training procedure, based on the evidence framework, is used for model selection and to provide a credible interval on model predictions. The kernel survival analysis models are found to be more accurate than models based on more traditional survival analysis techniques but also suggest a risk assessment of the foodborne botulism hazard would benefit from the collection of additional data. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neuromorphic depth-from-motion vision model with STDP adaptation

    Page(s): 482 - 495
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1066 KB)  

    We propose a simplified depth-from-motion vision model based on leaky integrate-and-fire (LIF) neurons for edge detection and two-dimensional depth recovery. In the model, every LIF neuron is able to detect the irradiance edges passing through its receptive field in an optical flow field, and respond to the detection by firing a spike when the neuron's firing criterion is satisfied. If a neuron fires a spike, the time-of-travel of the spike-associated edge is transferred as the prediction information to the next synapse-linked neuron to determine its state. Correlations between input spikes and their timing thus encode depth in the visual field. The adaptation of synapses mediated by spike-timing-dependent plasticity is used to improve the algorithm's robustness against inaccuracy caused by spurious edge propagation. The algorithm is characterized on both artificial and real image sequences. The implementation of the algorithm in analog very large scale integrated (aVLSI) circuitry is also discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neuromorphic walking gait control

    Page(s): 496 - 508
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1014 KB)  

    We present a neuromorphic pattern generator for controlling the walking gaits of four-legged robots which is inspired by central pattern generators found in the nervous system and which is implemented as a very large scale integrated (VLSI) chip. The chip contains oscillator circuits that mimic the output of motor neurons in a strongly simplified way. We show that four coupled oscillators can produce rhythmic patterns with phase relationships that are appropriate to generate all four-legged animal walking gaits. These phase relationships together with frequency and duty cycle of the oscillators determine the walking behavior of a robot driven by the chip, and they depend on a small set of stationary bias voltages. We give analytic expressions for these dependencies. This chip reduces the complex, dynamic inter-leg control problem associated with walking gait generation to the problem of setting a few stationary parameters. It provides a compact and low power solution for walking gait control in robots. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive neural network control for a class of low-triangular-structured nonlinear systems

    Page(s): 509 - 514
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (294 KB) |  | HTML iconHTML  

    In this paper, a class of unknown perturbed nonlinear systems is theoretically stabilized by using adaptive neural network control. The systems, with disturbances and nonaffine unknown functions, have low triangular structure, which generalizes both strict-feedback uncertain systems and pure-feedback ones. There do not exist any effective methods to stabilize this kind of systems. With some new conclusions for Nussbaum-Gain functions (NGF) and the idea of backstepping, semiglobal, uniformal, and ultimate boundedness of all the signals in the closed-loop is proved at equilibrium point. The two problems, control directions and control singularity, are well dealt with. The effectiveness of proposed scheme is shown by simulation on a proper nonlinear system. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Computation of Adalines' sensitivity to weight perturbation

    Page(s): 515 - 519
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (222 KB) |  | HTML iconHTML  

    In this paper, the sensitivity of Adalines to weight perturbation is discussed. According to the discrete feature of Adalines' input and output, the sensitivity is defined as the probability of an Adaline's erroneous outputs due to weight perturbation with respect to all possible inputs. By means of hypercube model and analytical geometry method, a heuristic algorithm is given to accurately compute the sensitivity. The accuracy of the algorithm is verified by computer simulations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Associative memory design for 256 gray-level images using a multilayer neural network

    Page(s): 519 - 522
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (606 KB) |  | HTML iconHTML  

    A design procedure is presented for neural associative memories storing gray-scale images. It is an evolution of a previous work based on the decomposition of the image with 2L gray levels into L binary patterns, stored in L uncoupled neural networks. In this letter, an L-layer neural network is proposed with both intralayer and interlayer connections. The connections between different layers introduce interactions among all the neurons, increasing the recall performance with respect to the uncoupled case. In particular, the proposed network can store images with the commonly used number of 256 gray levels instead of 16, as in the previous approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Convergence of gradient method with momentum for two-Layer feedforward neural networks

    Page(s): 522 - 525
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (152 KB) |  | HTML iconHTML  

    A gradient method with momentum for two-layer feedforward neural networks is considered. The learning rate is set to be a constant and the momentum factor an adaptive variable. Both the weak and strong convergence results are proved, as well as the convergence rates for the error function and for the weight. Compared to the existing convergence results, our results are more general since we do not require the error function to be quadratic. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the new method for the control of discrete nonlinear dynamic systems using neural networks

    Page(s): 526 - 529
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (163 KB) |  | HTML iconHTML  

    This correspondence points out an incorrect statement in Adetona et al., 2000, and Adetona et al., 2004, about the application of the proposed control law to nonminimum phase systems. A counterexample shows the limitations of the control law and, furthermore, its control capability to nonminimum phase systems is explained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope