By Topic

Neural Networks, IEEE Transactions on

Issue 3 • Date May 1996

Filter Results

Displaying Results 1 - 25 of 25
  • Fundamentals of Artificial Neural Networks [Book Reviews]

    Publication Year: 1996
    Save to Project icon | Request Permissions | PDF file iconPDF (267 KB)  
    Freely Available from IEEE
  • Evolutionary learning of nearest-neighbor MLP

    Publication Year: 1996 , Page(s): 762 - 767
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (540 KB)  

    The nearest-neighbor multilayer perceptron (NN-MLP) is a single-hidden-layer network suitable for pattern recognition. To design an NN-MLP efficiently, this paper proposes a new evolutionary algorithm consisting of four basic operations: recognition, remembrance, reduction, and review. Experimental results show that this algorithm can produce the smallest or nearly smallest networks from random initial ones View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Codeword distribution for frequency sensitive competitive learning with one-dimensional input data

    Publication Year: 1996 , Page(s): 752 - 756
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (356 KB)  

    We study the codeword distribution for a conscience-type competitive learning algorithm, frequency sensitive competitive learning (FSCL), using one-dimensional input data. We prove that the asymptotic codeword density in the limit of large number of codewords is given by a power law of the form Q(x)=C·P(x)α, where P(x) is the input data density and α depends on the algorithm and the form of the distortion measure to be minimized. We further show that the algorithm can be adjusted to minimize any Lp distortion measure with p ranging in (0,2] View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Synchronization and desynchronization in a network of locally coupled Wilson-Cowan oscillators

    Publication Year: 1996 , Page(s): 541 - 554
    Cited by:  Papers (29)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1384 KB)  

    A network of Wilson-Cowan (WC) oscillators is constructed, and its emergent properties of synchronization and desynchronization are investigated by both computer simulation and formal analysis. The network is a 2D matrix, where each oscillator is coupled only to its neighbors. We show analytically that a chain of locally coupled oscillators (the piecewise linear approximation to the WC oscillator) synchronizes, and we present a technique to rapidly entrain finite numbers of oscillators. The coupling strengths change on a fast time scale based on a Hebbian rule. A global separator is introduced which receives input from and sends feedback to each oscillator in the matrix. The global separator is used to desynchronize different oscillator groups. Unlike many other models, the properties of this network emerge from local connections that preserve spatial relationships among components and are critical for encoding Gestalt principles of feature grouping. The ability to synchronize and desynchronize oscillator groups within this network offers a promising approach for pattern segmentation and figure/ground segregation based on oscillatory correlation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Randomness in generalization ability: a source to improve it

    Publication Year: 1996 , Page(s): 676 - 685
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1076 KB)  

    Among several models of neurons and their interconnections, feedforward artificial neural networks (FFANNs) are most popular, because of their simplicity and effectiveness. Difficulties such as long learning time and local minima may not affect FFANNs as much as the question of generalization ability, because a network needs only one training, and then it may be used for a long time. This paper reports our observations about randomness in generalization ability of FFANNs. A novel method for measuring generalization ability is defined. This method can be used to identify degree of randomness in generalization ability of learning systems. If an FFANN architecture shows randomness in generalization ability for a given problem, multiple networks can be used to improve it. We have developed a model, called voting model, for predicting generalization ability of multiple networks. It has been shown that if correct classification probability of a single network is greater than half, then as the number of networks in a voting network is increased so does its generalization ability. Further analysis has shown that VC-dimension of the voting network model may increase monotonically as the number of networks in the voting networks is increased View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Traffic management of a satellite communication network using stochastic optimization

    Publication Year: 1996 , Page(s): 732 - 744
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1128 KB)  

    The performance of nonhierarchical circuit switched networks at moderate load conditions is improved when alternate routes are made available. Alternate routes, however, introduce instability under heavy and overloaded conditions, and under these load conditions network performance is found to deteriorate. To alleviate this problem, a control mechanism is used where, a fraction of the capacity of each link is reserved for direct routed calls. In this work, a traffic management scheme is developed to enhance the performance of a mesh-connected, circuit-switched satellite communication network. The network load is measured and the network is continually adapted by reconfiguring the map to suit the current traffic conditions. The routing is performed dynamically. The reconfiguration of the network is done by properly allocating the capacity of each link and placing an optimal reservation on each link. The optimization is done by using two neural network-based optimization techniques: simulated annealing and mean field annealing. A comparative study is done between these two techniques. The results from the simulation study show that this method of traffic management performs better than the pure dynamic routing with a fixed configuration View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A coupled gradient network approach for static and temporal mixed-integer optimization

    Publication Year: 1996 , Page(s): 578 - 593
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1508 KB)  

    Utilizes the ideas of artificial neural networks to propose new solution methods for a class of constrained mixed-integer optimization problems. These new solution methods are more suitable to parallel implementation than the usual sequential methods of mathematical programming. Another attractive feature of the proposed approach is that some global search mechanisms may be easily incorporated into the computation, producing results which are more globally optimal. To formulate the solution method proposed in this paper, a penalty function approach is used to define a coupled gradient-type network with an appropriate architecture, energy function and dynamics such that high-quality solutions may be obtained upon convergence of the dynamics. Finally, it is shown how the coupled gradient net may be extended to handle temporal mixed-integer optimization problems, and simulations are presented which demonstrate the effectiveness of the approach View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Extending the functional equivalence of radial basis function networks and fuzzy inference systems

    Publication Year: 1996 , Page(s): 776 - 781
    Cited by:  Papers (46)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (528 KB)  

    We establish the functional equivalence of a generalized class of Gaussian radial basis function (RBFs) networks and the full Takagi-Sugeno model (1983) of fuzzy inference. This generalizes an existing result which applies to the standard Gaussian RBF network and a restricted form of the Takagi-Sugeno fuzzy system. The more general framework allows the removal of some of the restrictive conditions of the previous result View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Face recognition using artificial neural network group-based adaptive tolerance (GAT) trees

    Publication Year: 1996 , Page(s): 555 - 567
    Cited by:  Papers (23)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1408 KB)  

    Recent artificial neural network research has focused on simple models, but such models have not been very successful in describing complex systems (such as face recognition). This paper introduces the artificial neural network group-based adaptive tolerance (GAT) tree model for translation-invariant face recognition, suitable for use in an airport security system. GAT trees use a two-stage divide-and-conquer tree-type approach. The first stage determines general properties of the input, such as whether the facial image contains glasses or a beard. The second stage identifies the individual. Face perception classification, detection of front faces with glasses and/or beards, and face recognition results using GAT trees under laboratory conditions are presented. We conclude that the neural network group-based model offers significant improvement over conventional neural network trees for this task View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A high-performance neural network for solving linear and quadratic programming problems

    Publication Year: 1996 , Page(s): 643 - 651
    Cited by:  Papers (30)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (592 KB)  

    Two classes of high-performance neural networks for solving linear and quadratic programming problems are given. We prove that the new system converges globally to the solutions of the linear and quadratic programming problems. In a neural network, network parameters are usually not specified. The proposed models can overcome numerical difficulty caused by neural networks with network parameters and obtain desired approximate solutions of the linear and quadratic programming problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A categorizing associative memory using an adaptive classifier and sparse coding

    Publication Year: 1996 , Page(s): 669 - 675
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (708 KB)  

    This paper proposes a neural network that stores and retrieves sparse patterns categorically, the patterns being random realizations of a sequence of biased (0,1) Bernoulli trials. The neural network, denoted as categorizing associative memory, consists of two modules: 1) an adaptive classifier (AC) module that categorizes input data; and 2) an associative memory (AM) module that stores input patterns in each category according to a Hebbian learning rule, after the AC module has stabilized its learning of that category. We show that during training of the AC module, the weights in the AC module belonging to a category converge to the probability of a “1” occurring in a pattern from that category. This fact is used to set the thresholds of the AM module optimally without requiring any a priori knowledge about the stored patterns View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalized deterministic annealing

    Publication Year: 1996 , Page(s): 686 - 699
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1672 KB)  

    We develop a general formalism for computing high quality, low-cost solutions to nonconvex combinatorial optimization problems expressible as distributed interacting local constraints. For problems of this type, generalized deterministic annealing (GDA) avoids the performance-related sacrifices of current techniques. GDA exploits the localized structure of such problems by assigning K-state neurons to each optimization variable. The neuron values correspond to the probability densities of K-state local Markov chains and may be updated serially or in parallel; the Markov model is derived from the Markov model of simulated annealing (SA), although it is greatly simplified. Theorems are presented that firmly establish the convergence properties of GDA, as well as supplying practical guidelines for selecting the initial and final temperatures in the annealing process. A benchmark image enhancement application is provided where the performance of GDA is compared to other optimization methods. The empirical data taken in conjunction with the formal analytical results suggest that GDA enjoys significant performance advantages relative to current methods for combinatorial optimization View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Incremental backpropagation learning networks

    Publication Year: 1996 , Page(s): 757 - 761
    Cited by:  Papers (26)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (480 KB)  

    How to learn new knowledge without forgetting old knowledge is a key issue in designing an incremental-learning neural network. In this paper, we present a new incremental learning method for pattern recognition, called the “incremental backpropagation learning network”, which employs bounded weight modification and structural adaptation learning rules and applies initial knowledge to constrain the learning process. The viability of this approach is demonstrated for classification problems including the iris and the promoter domains View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A training rule which guarantees finite-region stability for a class of closed-loop neural-network control systems

    Publication Year: 1996 , Page(s): 745 - 751
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (584 KB)  

    A training method for a class of neural network controllers is presented which guarantees closed-loop system stability. The controllers are assumed to be nonlinear, feedforward, sampled-data, full-state regulators implemented as single hidden-layer neural networks. The controlled systems must be locally hermitian and observable. Stability of the closed-loop system is demonstrated by determining a Lyapunov function, which can be used to identify a finite stability region about the regulator point View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Equilibrium capacity of analog feedback neural networks

    Publication Year: 1996 , Page(s): 782 - 787
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (416 KB)  

    A method for estimating the equilibrium capacity of a general class of analog feedback neural networks is presented in this brief paper. Some explicit relationships between upper bound of the number of possible stable equilibria and the network parameters such as self-feedback coefficients, weights, and gains of a feedback neural network are obtained. Increasing the equilibrium capacity using multimodal sigmoidal functions is also discussed. Some examples are provided to demonstrate the effectiveness of the analytical results presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Entropy maximization networks: an application to breast cancer prognosis

    Publication Year: 1996 , Page(s): 568 - 577
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1020 KB)  

    Describes two artificial neural network architectures for constructing maximum entropy models using multinomial distributions. The architectures presented maximize entropy in two ways: by the use of the partition function (which involves the solution of simultaneous polynomial equations), and by constrained gradient ascent. Results comparing the convergence properties of these two architectures are presented. The practical use of these two architectures as a method of inference is illustrated by an application to the prediction of metastases in early breast cancer patients. To assess the predictive accuracy of the maximum entropy models, we compared the results with those obtained by the use of the multilayer perceptron and the probabilistic neural network View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Reinforcement learning for an ART-based fuzzy adaptive learning control network

    Publication Year: 1996 , Page(s): 709 - 731
    Cited by:  Papers (25)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2208 KB)  

    This paper proposes a reinforcement fuzzy adaptive learning control network (RFALCON), constructed by integrating two fuzzy adaptive learning control networks (FALCON), each of which has a feedforward multilayer network and is developed for the realization of a fuzzy controller. One FALCON performs as a critic network (fuzzy predictor), the other as an action network (fuzzy controller). Using temporal difference prediction, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network performs a stochastic exploratory algorithm to adapt itself according to the internal reinforcement signal. An ART-based reinforcement structure/parameter-learning algorithm is developed for constructing the RFALCON dynamically. During the learning process, structure and parameter learning are performed simultaneously. RFALCON can construct a fuzzy control system through a reward/penalty signal. It has two important features; it reduces the combinatorial demands of system adaptive linearization, and it is highly autonomous View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Local linear perceptrons for classification

    Publication Year: 1996 , Page(s): 788 - 794
    Cited by:  Papers (34)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (472 KB)  

    A structure composed of local linear perceptrons for approximating global class discriminants is investigated. Such local linear models may be combined in a cooperative or competitive way. In the cooperative model, a weighted sum of the outputs of the local perceptrons is computed where the weight is a function of the distance between the input and the position of the local perceptron. In the competitive model, the cost function dictates a mixture model where only one of the local perceptrons give output. Learning of the local models' positions and the linear mappings they implement are coupled and both supervised. We show that this is preferable to the uncoupled case where the positions are trained in an unsupervised manner before the separate, supervised training of mappings. We use goodness criteria based on the cross-entropy and give learning equations for both the cooperative and competitive cases. The coupled and uncoupled versions of cooperative and competitive approaches are compared among themselves and with multilayer perceptrons of sigmoidal hidden units and radial basis functions (RBFs) of Gaussian units on the application of recognition of handwritten digits. The criteria of comparison are the generalization accuracy, learning time, and the number of free parameters. We conclude that even on such a high-dimensional problem, such local models are promising. They generalize much better than RBF's and use much less memory. When compared with multilayer perceptrons, we note that local models learn much faster and generalize as well and sometimes better with comparable number of parameters View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Harmonic competition: a self-organizing multiple criteria optimization

    Publication Year: 1996 , Page(s): 652 - 668
    Cited by:  Papers (12)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1356 KB)  

    Harmonic competition is a learning strategy based upon winner-take-all or winner-take-quota with respect to a composite of heterogeneous subcosts. This learning is unsupervised and organizes itself. The subcosts may conflict with each other. Thus, the total learning system realizes a self-organizing multiple criteria optimization. The subcosts are combined additively and multiplicatively using adjusting parameters. For such a total cost, a general successive learning algorithm is derived first. Then, specific problems in the Euclidian space are addressed. Vector quantization with various constraints and traveling salesperson problems are selected as test problems. The former is a typical class of problems where the number of neurons is less than that of the data. The latter is an opposite case. Duality exists in these two classes. In both cases, the combination parameters of the subcosts show wide dynamic ranges in the course of learning. It is possible, however, to decide the parameter control from the structure of the total cost. This method finds a preferred solution from the Pareto optimal set of the multiple object optimization. Controlled mutations motivated by genetic algorithms are proved to be effective in finding near-optimal solutions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Shot-noise-limited performance of optical neural networks

    Publication Year: 1996 , Page(s): 700 - 708
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (844 KB)  

    The performance of neural networks for which weights and signals are modeled by shot-noise processes is considered. Examples of such networks are optical neural networks and biological systems. We develop a theory that facilitates the computation of the average probability of error in binary-input/binary-output multistage and recurrent networks. We express the probability of error in terms of two key parameters: the computing-noise parameter and the weight-recording-noise parameter. The former is the average number of particles per clock cycle per signal and it represents noise due to the particle nature of the signal. The latter represents noise in the weight-recording process and is the average number of particles per weight. For a fixed computing-noise parameter, the probability of error decreases with the increase in the recording-noise parameter and saturates at a level limited by the computing-noise parameter. A similar behavior is observed when the role of the two parameters is interchanged. As both parameters increase, the probability of error decreases to zero exponentially fast at a rate that is determined using large deviations. We show that the performance can be optimized by a selective choice of the nonlinearity threshold levels. For recurrent networks, as the number of iterations increases, the probability of error increases initially and then saturates at a level determined by the stationary distribution of a Markov chain View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A fast multilayer neural-network training algorithm based on the layer-by-layer optimizing procedures

    Publication Year: 1996 , Page(s): 768 - 775
    Cited by:  Papers (21)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (568 KB)  

    A faster new learning algorithm to adjust the weights of the multilayer feedforward neural network is proposed. In this new algorithm, the weight matrix (W2) of the output layer and the output vector (Y) of the previous layer are treated as two variable sets. An optimal solution pair (W2*,YP*) is found to minimize the sum-square-error of the patterns input. YP* is then used as the desired output of the previous layer. The optimal weight matrix and layer output vector of the hidden layers in the network is found with the same method as that used for the output layer. In addition, the dynamic forgetting factors method makes the proposed new algorithm even more powerful in dynamic system identification. Computer simulation shows that the new algorithm outmatches other learning algorithms both in converging speed and in computation time required View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An approach to stability criteria of neural-network control systems

    Publication Year: 1996 , Page(s): 629 - 642
    Cited by:  Papers (43)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (752 KB)  

    This paper discusses stability of neural network (NN)-based control systems using Lyapunov approach. First, it is pointed out that the dynamics of NN systems can be represented by a class of nonlinear systems treated as linear differential inclusions (LDI). Next, stability conditions for the class of nonlinear systems are derived and applied to the stability analysis of single NN systems and feedback NN control systems. Furthermore, a method of parameter region (PR) representation, which graphically shows the location of parameters of nonlinear systems, is proposed by introducing new concepts of vertex point and minimum representation. From these concepts, an important theorem, which is useful for effectively finding a Lyapunov function, is derived. Stability criteria of single NN systems are illustrated in terms of PR representation. Finally, stability of feedback NN control systems, which consist of a plant represented by an NN and an NN controller, is analyzed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Existence and stability of a unique equilibrium in continuous-valued discrete-time asynchronous Hopfield neural networks

    Publication Year: 1996 , Page(s): 620 - 628
    Cited by:  Papers (19)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (956 KB)  

    It is shown that the assumption of D-stability of the interconnection matrix, together with the standard assumptions on the activation functions, guarantee the existence of a unique equilibrium under a synchronous mode of operation as well as a class of asynchronous modes. For the synchronous mode, these assumptions are also shown to imply local asymptotic stability of the equilibrium. For the asynchronous mode of operation, two results are derived. First, it is shown that symmetry and stability of the interconnection matrix guarantee local asymptotic stability of the equilibrium under a class of asynchronous modes-this is referred to as local absolute asymptotic stability. Second, it is shown that, under the standard assumptions, if the nonnegative matrix whose elements are the absolute values of the corresponding elements of the interconnection matrix is stable, then the equilibrium is globally absolutely asymptotically stable under a class of asynchronous modes. The results obtained are discussed from the points of view of their applications, robustness, and their relationship to earlier results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Worst-case quadratic loss bounds for prediction using linear functions and gradient descent

    Publication Year: 1996 , Page(s): 604 - 619
    Cited by:  Papers (28)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1300 KB)  

    Studies the performance of gradient descent (GD) when applied to the problem of online linear prediction in arbitrary inner product spaces. We prove worst-case bounds on the sum of the squared prediction errors under various assumptions concerning the amount of a priori information about the sequence to predict. The algorithms we use are variants and extensions of online GD. Whereas our algorithms always predict using linear functions as hypotheses, none of our results requires the data to be linearly related. In fact, the bounds proved on the total prediction loss are typically expressed as a function of the total loss of the best fixed linear predictor with bounded norm. All the upper bounds are tight to within constants. Matching lower bounds are provided in some cases. Finally, we apply our results to the problem of online prediction for classes of smooth functions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Parallel distributed detection of feature trajectories in multiple discontinuous motion image sequences

    Publication Year: 1996 , Page(s): 594 - 603
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2056 KB)  

    Concerns the 3D interpretation of image sequences showing multiple objects in motion. Each object exhibits smooth motion except at certain time instants when a motion discontinuity may occur. The objects are assumed to contain point features which are detected as the images are acquired. Estimating feature trajectories in the first two frames amounts to feature matching. As more images are acquired, existing trajectories are extended. Both initial detection and extension of trajectories are done by enforcing pertinent constraints from among the following: similarity of the image plane arrangement of neighboring features, smoothness of the 3D motion and smoothness of the image plane motion. The constraints are incorporated into energy functions which are minimized using 2D Hopfield networks. Wrong matches that result from convergence to local minima are eliminated using a 1D Hopfield-like network. Experimental results on several image sequences are shown View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope