By Topic

Neural Networks, IEEE Transactions on

Issue 2 • Date March 2005

Filter Results

Displaying Results 1 - 25 of 30
  • Table of contents

    Page(s): c1 - c4
    Save to Project icon | Request Permissions | PDF file iconPDF (41 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): c2
    Save to Project icon | Request Permissions | PDF file iconPDF (35 KB)  
    Freely Available from IEEE
  • Encoding strategy for maximum noise tolerance bidirectional associative memory

    Page(s): 293 - 300
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (412 KB) |  | HTML iconHTML  

    In this paper, the basic bidirectional associative memory (BAM) is extended by choosing weights in the correlation matrix, for a given set of training pairs, which result in a maximum noise tolerance set for BAM. We prove that for a given set of training pairs, the maximum noise tolerance set is the largest, in the sense that this optimized BAM will recall the correct training pair if any input pattern is within the maximum noise tolerance set and at least one pattern outside the maximum noise tolerance set by one Hamming distance will not converge to the correct training pair. This maximum tolerance set is the union of the maximum basins of attraction. A standard genetic algorithm (GA) is used to calculate the weights to maximize the objective function which generates a maximum tolerance set for BAM. Computer simulations are presented to illustrate the error correction and fault tolerance properties of the optimized BAM. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability criteria for unsupervised temporal association networks

    Page(s): 301 - 311
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (512 KB) |  | HTML iconHTML  

    A biologically realizable, unsupervised learning rule is described for the online extraction of object features, suitable for solving a range of object recognition tasks. Alterations to the basic learning rule are proposed which allow the rule to better suit the parameters of a given input space. One negative consequence of such modifications is the potential for learning instability. The criteria for such instability are modeled using digital filtering techniques and predicted regions of stability and instability tested. The result is a family of learning rules which can be tailored to the specific environment, improving both convergence times and accuracy over the standard learning rule, while simultaneously insuring learning stability. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities

    Page(s): 312 - 324
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (400 KB) |  | HTML iconHTML  

    It is well known that Boltzmann machines are nonregular statistical models. The set of their parameters for a small size model is an analytic set with singularities in the space of a large size one. The mathematical foundation of their learning is not yet constructed because of these singularities, though they are applied to information engineering. Recently we established a method to calculate the Bayes generalization errors using an algebraic geometric method even if the models are nonregular. This paper clarifies that the upper bounds of generalization errors in Boltzmann machines are smaller than those in regular statistical models. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Linear-least-squares initialization of multilayer perceptrons through backpropagation of the desired response

    Page(s): 325 - 337
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (704 KB) |  | HTML iconHTML  

    Training multilayer neural networks is typically carried out using descent techniques such as the gradient-based backpropagation (BP) of error or the quasi-Newton approaches including the Levenberg-Marquardt algorithm. This is basically due to the fact that there are no analytical methods to find the optimal weights, so iterative local or global optimization techniques are necessary. The success of iterative optimization procedures is strictly dependent on the initial conditions, therefore, in this paper, we devise a principled novel method of backpropagating the desired response through the layers of a multilayer perceptron (MLP), which enables us to accurately initialize these neural networks in the minimum mean-square-error sense, using the analytic linear least squares solution. The generated solution can be used as an initial condition to standard iterative optimization algorithms. However, simulations demonstrate that in most cases, the performance achieved through the proposed initialization scheme leaves little room for further improvement in the mean-square-error (MSE) over the training set. In addition, the performance of the network optimized with the proposed approach also generalizes well to testing data. A rigorous derivation of the initialization algorithm is presented and its high performance is verified with a number of benchmark training problems including chaotic time-series prediction, classification, and nonlinear system identification with MLPs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Pareto evolutionary neural networks

    Page(s): 338 - 354
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (952 KB) |  | HTML iconHTML  

    For the purposes of forecasting (or classification) tasks neural networks (NNs) are typically trained with respect to Euclidean distance minimization. This is commonly the case irrespective of any other end user preferences. In a number of situations, most notably time series forecasting, users may have other objectives in addition to Euclidean distance minimization. Recent studies in the NN domain have confronted this problem by propagating a linear sum of errors. However this approach implicitly assumes a priori knowledge of the error surface defined by the problem, which, typically, is not the case. This study constructs a novel methodology for implementing multiobjective optimization within the evolutionary neural network (ENN) domain. This methodology enables the parallel evolution of a population of ENN models which exhibit estimated Pareto optimality with respect to multiple error measures. A new method is derived from this framework, the Pareto evolutionary neural network (Pareto-ENN). The Pareto-ENN evolves a population of models that may be heterogeneous in their topologies inputs and degree of connectivity, and maintains a set of the Pareto optimal ENNs that it discovers. New generalization methods to deal with the unique properties of multiobjective error minimization that are not apparent in the uni-objective case are presented and compared on synthetic data, with a novel method based on bootstrapping of the training data shown to significantly improve generalization ability. Finally experimental evidence is presented in this study demonstrating the general application potential of the framework by generating populations of ENNs for forecasting 37 different international stock indexes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Self-organizing learning array

    Page(s): 355 - 363
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (910 KB) |  | HTML iconHTML  

    A new machine learning concept-self-organizing learning array (SOLAR)-is presented. It is a sparsely connected, information theory-based learning machine, with a multilayer structure. It has reconfigurable processing units (neurons) and an evolvable system structure, which makes it an adaptive classification system for a variety of machine learning problems. Its multilayer structure can handle complex problems. Based on the entropy estimation, information theory-based learning is performed locally at each neuron. Neural parameters and connections that correspond to minimum entropy are adaptively set for each neuron. By choosing connections for each neuron, the system sets up its wiring and completes its self-organization. SOLAR classifies input data based on the weighted statistical information from all the neurons. The system classification ability has been simulated and experiments were conducted using test-bench data. Results show a very good performance compared to other classification methods. An important advantage of this structure is its scalability to a large system and ease of hardware implementation on regular arrays of cells. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis of augmented-input-Layer RBFNN

    Page(s): 364 - 369
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (303 KB) |  | HTML iconHTML  

    In this paper we present and analyze a new structure for designing a radial basis function neural network (RBFNN). In the training phase, input layer of RBFNN is augmented with desired output vector. Generalization phase involves the following steps: 1) identify the cluster to which a previously unseen input vector belongs; 2) augment the input layer with an average of the targets of the input vectors in the identified cluster; and 3) use the augmented network to estimate the unknown target. It is shown that, under some reasonable assumptions, the generalization error function admits an upper bound in terms of the quantization errors minimized when determining the centers of the proposed method over the training set and the difference between training samples and generalization samples in a deterministic setting. When the difference between the training and generalization samples goes to zero, the upper bound can be made arbitrarily small by increasing the number of hidden neurons. Computer simulations verified the effectiveness of the proposed method. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An energy function-based design method for discrete hopfield associative memory with attractive fixed points

    Page(s): 370 - 378
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (392 KB) |  | HTML iconHTML  

    An energy function-based autoassociative memory design method to store a given set of unipolar binary memory vectors as attractive fixed points of an asynchronous discrete Hopfield network (DHN) is presented. The discrete quadratic energy function whose local minima correspond to the attractive fixed points of the network is constructed via solving a system of linear inequalities derived from the strict local minimality conditions. The weights and the thresholds are then calculated using this energy function. If the inequality system is infeasible, we conclude that no such asynchronous DHN exists, and extend the method to design a discrete piecewise quadratic energy function, which can be minimized by a generalized version of the conventional DHN, also proposed herein. In spite of its computational complexity, computer simulations indicate that the original method performs better than the conventional design methods in the sense that the memory can store, and provide the attractiveness for almost all memory sets whose cardinality is less than or equal to the dimension of its elements. The overall method, together with its extension, guarantees the storage of an arbitrary collection of memory vectors, which are mutually at least two Hamming distances away from each other, in the resulting network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A recurrent neural network for solving nonlinear convex programs subject to linear constraints

    Page(s): 379 - 386
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (826 KB) |  | HTML iconHTML  

    In this paper, we propose a recurrent neural network for solving nonlinear convex programming problems with linear constraints. The proposed neural network has a simpler structure and a lower complexity for implementation than the existing neural networks for solving such problems. It is shown here that the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal solution within a finite time under the condition that the objective function is strictly convex. Compared with the existing convergence results, the present results do not require Lipschitz continuity condition on the objective function. Finally, examples are provided to show the applicability of the proposed neural network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Passivity-based neural network adaptive output feedback control for nonlinear nonnegative dynamical systems

    Page(s): 387 - 398
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (501 KB) |  | HTML iconHTML  

    The potential clinical applications of adaptive neural network control for pharmacology in general, and anesthesia and critical care unit medicine in particular, are clearly apparent. Specifically, monitoring and controlling the depth of anesthesia in surgery is of particular importance. Nonnegative and compartmental models provide a broad framework for biological and physiological systems, including clinical pharmacology, and are well suited for developing models for closed-loop control of drug administration. In this paper, we develop a neural adaptive output feedback control framework for adaptive set-point regulation of nonlinear uncertain nonnegative and compartmental systems. The proposed framework is Lyapunov-based and guarantees ultimate boundedness of the error signals corresponding to the physical system states and the neural network weighting gains. The approach is applicable to nonlinear nonnegative systems with unmodeled dynamics of unknown dimension and guarantees that the physical system states remain in the nonnegative orthant of the state-space for nonnegative initial conditions. Finally, a numerical example involving the infusion of the anesthetic drug midazolam for maintaining a desired constant level of depth of anesthesia for noncardiac surgery is provided to demonstrate the efficacy of the proposed approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural network adaptive control for nonlinear nonnegative dynamical systems

    Page(s): 399 - 413
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (700 KB) |  | HTML iconHTML  

    Nonnegative and compartmental dynamical system models are derived from mass and energy balance considerations that involve dynamic states whose values are nonnegative. These models are widespread in engineering and life sciences and typically involve the exchange of nonnegative quantities between subsystems or compartments wherein each compartment is assumed to be kinetically homogeneous. In this paper, we develop a full-state feedback neural adaptive control framework for adaptive set-point regulation of nonlinear uncertain nonnegative and compartmental systems. The proposed framework is Lyapunov-based and guarantees ultimate boundedness of the error signals corresponding to the physical system states and the neural network weighting gains. In addition, the neural adaptive controller guarantees that the physical system states remain in the nonnegative orthant of the state-space for nonnegative initial conditions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Direct adaptive controller for nonaffine nonlinear systems using self-structuring neural networks

    Page(s): 414 - 422
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (601 KB) |  | HTML iconHTML  

    A direct adaptive state-feedback controller is proposed for highly nonlinear systems. We consider uncertain or ill-defined nonaffine nonlinear systems and employ a neural network (NN) with flexible structure, i.e., an online variation of the number of neurons. The NN approximates and adaptively cancels an unknown plant nonlinearity. A control law and adaptive laws for the weights in the hidden layer and output layer of the NN are established so that the whole closed-loop system is stable in the sense of Lyapunov. Moreover, the tracking error is guaranteed to be uniformly asymptotically stable (UAS) rather than uniformly ultimately bounded (UUB) with the aid of an additional robustifying control term. The proposed control algorithm is relatively simple and requires no restrictive conditions on the design constants for the stability. The efficiency of the proposed scheme is shown through the simulation of a simple nonaffine nonlinear system. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Soft learning vector quantization and clustering algorithms based on non-Euclidean norms: single-norm algorithms

    Page(s): 423 - 435
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (711 KB) |  | HTML iconHTML  

    This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on a weighted norm to measure the distance between the feature vectors and their prototypes. The development of LVQ and clustering algorithms is based on the minimization of a reformulation function under the constraint that the generalized mean of the norm weights be constant. According to the proposed formulation, the norm weights can be computed from the data in an iterative fashion together with the prototypes. An error analysis provides some guidelines for selecting the parameter involved in the definition of the generalized mean in terms of the feature variances. The algorithms produced from this formulation are easy to implement and they are almost as fast as clustering algorithms relying on the Euclidean norm. An experimental evaluation on four data sets indicates that the proposed algorithms outperform consistently clustering algorithms relying on the Euclidean norm and they are strong competitors to non-Euclidean algorithms which are computationally more demanding. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Face membership authentication using SVM classification tree generated by membership-based LLE data partition

    Page(s): 436 - 446
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1045 KB) |  | HTML iconHTML  

    This paper presents a new membership authentication method by face classification using a support vector machine (SVM) classification tree, in which the size of membership group and the members in the membership group can be changed dynamically. Unlike our previous SVM ensemble-based method, which performed only one face classification in the whole feature space, the proposed method employed a divide and conquer strategy that first performs a recursive data partition by membership-based locally linear embedding (LLE) data clustering, then does the SVM classification in each partitioned feature subset. Our experimental results show that the proposed SVM tree not only keeps the good properties that the SVM ensemble method has, such as a good authentication accuracy and the robustness to the change of members, but also has a considerable improvement on the stability under the change of membership group size. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multi-aspect target discrimination using hidden Markov models and neural networks

    Page(s): 447 - 459
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (566 KB) |  | HTML iconHTML  

    This paper presents a new multi-aspect pattern classification method using hidden Markov models (HMMs). Models are defined for each class, with the probability found by each model determining class membership. Each HMM model is enhanced by the use of a multilayer perception (MLP) network to generate emission probabilities. This hybrid system uses the MLP to find the probability of a state for an unknown pattern and the HMM to model the process underlying the state transitions. A new batch gradient descent-based method is introduced for optimal estimation of the transition and emission probabilities. A prediction method in conjunction with HMM model is also presented that attempts to improve the computation of transition probabilities by using the previous states to predict the next state. This method exploits the correlation information between consecutive aspects. These algorithms are then implemented and benchmarked on a multi-aspect underwater target classification problem using a realistic sonar data set collected in different bottom conditions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimizing the kernel in the empirical feature space

    Page(s): 460 - 474
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (624 KB) |  | HTML iconHTML  

    In this paper, we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space, an Euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved. Employing a data-dependent kernel, we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space. It is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by Cristianini. Extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data, and leads to a substantial, sometimes significant, improvement in the performance of various data classification algorithms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An intelligent adaptive control scheme for postsurgical blood pressure regulation

    Page(s): 475 - 483
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (426 KB) |  | HTML iconHTML  

    This paper presents an adaptive modeling and control scheme for drug delivery systems based on a generalized fuzzy neural network (G-FNN). The proposed G-FNN is a novel intelligent modeling tool, which can model unknown nonlinearities of complex drug delivery systems and adapt to changes and uncertainties in these systems online. It offers salient features, such as dynamic fuzzy neural topology, fast online learning ability and adaptability. System approximation formulated by the G-FNN is employed in the adaptive controller design for drug infusion in intensive care environment. In particular, this paper investigates automated regulation of mean arterial pressure (MAP) through intravenous infusion of sodium nitroprusside (SNP), which is one attractive application in automation of drug delivery. Simulation studies demonstrate the capability of the proposed approach in estimating the drug's effect and regulating blood pressure at a prescribed level. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • NFDTD concept

    Page(s): 484 - 490
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (488 KB) |  | HTML iconHTML  

    This paper combines artificial neural network (ANN) technique with the finite difference time domain (FDTD) technique. A detailed illustration of the concept, in this paper, uses a 3-8-1 feedforward artificial neural network (FF-ANN) for approximating the Z-component of the electric field in a rectangular waveguide in TM mode. The FDTD equation (i.e., the two-dimensional (2-D) wave equation in discrete form) is embedded into the cost function of the ANN. Results of implementing this technique in a one-dimensional (1-D) transmission line resonator are also provided with 4-10-1 FF-ANN. The result of the leap-frog algorithm implementation, for this 1-D problem using a (3-6-1) × (3-6-1) hybrid FF-ANN, is also provided. The neural-finite difference time domain (NFDTD) results are compared with those of the traditional FDTD. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improved access to sequential motifs: a note on the architectural bias of recurrent networks

    Page(s): 491 - 494
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB) |  | HTML iconHTML  

    For many biological sequence problems the available data occupies only sparse regions of the problem space. To use machine learning effectively for the analysis of sparse data we must employ architectures with an appropriate bias. By experimentation we show that the bias of recurrent neural networks-recently analyzed by Tino et al. and Hammer and Tino-offers superior access to motifs (sequential patterns) compared to the, in bioinformatics, standardly used feedforward neural networks. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A spatially constrained mixture model for image segmentation

    Page(s): 494 - 498
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (532 KB) |  | HTML iconHTML  

    Gaussian mixture models (GMMs) constitute a well-known type of probabilistic neural networks. One of their many successful applications is in image segmentation, where spatially constrained mixture models have been trained using the expectation-maximization (EM) framework. In this letter, we elaborate on this method and propose a new methodology for the M-step of the EM algorithm that is based on a novel constrained optimization formulation. Numerical experiments using simulated images illustrate the superior performance of our method in terms of the attained maximum value of the objective function and segmentation accuracy compared to previous implementations of this approach. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An improved conjugate gradient scheme to the solution of least squares SVM

    Page(s): 498 - 501
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (366 KB) |  | HTML iconHTML  

    The least square support vector machines (LS-SVM) formulation corresponds to the solution of a linear system of equations. Several approaches to its numerical solutions have been proposed in the literature. In this letter, we propose an improved method to the numerical solution of LS-SVM and show that the problem can be solved using one reduced system of linear equations. Compared with the existing algorithm for LS-SVM, the approach used in this letter is about twice as efficient. Numerical results using the proposed method are provided for comparisons with other existing algorithms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Solving systems of linear equations via gradient systems with discontinuous righthand sides: application to LS-SVM

    Page(s): 501 - 505
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (208 KB) |  | HTML iconHTML  

    A gradient system with discontinuous righthand side that solves an underdetermined system of linear equations in the L1 norm is presented. An upper bound estimate for finite time convergence to a solution set of the system of linear equations is shown by means of the Persidskii form of the gradient system and the corresponding nonsmooth diagonal type Lyapunov function. This class of systems can be interpreted as a recurrent neural network and an application devoted to solving least squares support vector machines (LS-SVM) is used as an example. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Refractory pulse counting Processes in stochastic neural computers

    Page(s): 505 - 508
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (148 KB)  

    This letter quantitatively investigates the effect of a temporary refractory period or dead time in the ability of a stochastic Bernoulli processor to record subsequent pulse events, following the arrival of a pulse. These effects can arise in either the input detectors of a stochastic neural network or in subsequent processing. A transient period is observed, which increases with both the dead time and the Bernoulli probability of the dead-time free system, during which the system reaches equilibrium. Unless the Bernoulli probability is small compared to the inverse of the dead time, the mean and variance of the pulse count distributions are both appreciably reduced. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope