Scheduled System Maintenance on May 29th, 2015:
IEEE Xplore will be upgraded between 11:00 AM and 10:00 PM EDT. During this time there may be intermittent impact on performance. We apologize for any inconvenience.
By Topic

Neural Networks, IEEE Transactions on

Issue 2 • Date March 1997

Filter Results

Displaying Results 1 - 25 of 30
  • On the "Identification and control of dynamical systems using neural networks"

    Publication Year: 1997
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    Referring to the above said paper by Narendra-Parthasarathy (ibid., vol.1, p4-27 (1990)), it is noted that the given Example 2 (p.15) has a third equilibrium state corresponding to the point (0.5, 0.5). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cascade ARTMAP: integrating neural computation and symbolic knowledge processing

    Publication Year: 1997 , Page(s): 237 - 250
    Cited by:  Papers (26)  |  Patents (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (383 KB)  

    This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Comments on "Stochastic choice of basis functions in adaptive function approximation and the functional-link net" [with reply]

    Publication Year: 1997 , Page(s): 452 - 454
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (52 KB)  

    This paper includes some comments and amendments of the above-mentioned paper by Igelnik et al. (1995). Subsequently, Theorem 1 in the above-mentioned paper has been revised. The significant change of the original theorem is the space of the thresholds in the hidden layer. The revised theorem says that the thresholds of hidden b/sub 0/, should be -w/sub 0//spl middot/y/sub 0/-u/sub 0/, where w/sub 0/=/spl alpha/w/spl circ//sub 0/; w/spl circ//sub 0/=(w/spl circ//sub 01/, /spl middot//spl middot//spl middot/, y/sub 0d/), and u/sub 0/ be independent and uniformly distributed in V/sup d/=[0; /spl Omega/]/spl times/[-/spl Omega/; /spl Omega/]/sup d-1/, I/sup d/, and [-2d/spl Omega/, 2d/spl Omega/], respectively. In reply, Igelnik et al. acknowledge that a factor of two was omitted in the statement of a trigonometric identity. However, the validity of the essential point of Theorem 1 is unaltered. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Correction to "A Modified HME Architecture for Text-Dependent Speaker Identification"

    Publication Year: 1997 , Page(s): 455
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (13 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recent Advances in Reinforcement Learning [Books in Brief]

    Publication Year: 1997 , Page(s): 456
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • Bayesian Learning for Neural Networks [Books in Brief]

    Publication Year: 1997 , Page(s): 456
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • A binary Hopfield neural-network approach for satellite broadcast scheduling problems

    Publication Year: 1997 , Page(s): 441 - 445
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (144 KB)  

    This paper presents a binary Hopfield neural network approach for finding a broadcasting schedule in a low-altitude satellite system. Our neural network is composed of simple binary neurons on the synchronous parallel computation, which is greatly suitable for implementation on a digital machine. With the help of heuristic methods, the neural network of a maximum of 200000 neurons can always find near-optimum solutions on a conventional workstation in our simulations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Iterative generation of higher-order nets in polynomial time using linear programming

    Publication Year: 1997 , Page(s): 402 - 412
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    This paper presents an algorithm for constructing and training a class of higher-order perceptrons for classification problems. The method uses linear programming models to construct and train the net. Its polynomial time complexity is proven and computational results are provided for several well-known problems. In all cases, very small nets were created compared to those reported in other computational studies View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability and statistical properties of second-order bidirectional associative memory

    Publication Year: 1997 , Page(s): 267 - 277
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (536 KB)  

    In this paper, a bidirectional associative memory (BAM) model with second-order connections, namely second-order bidirectional associative memory (SOBAM), is first reviewed. The stability and statistical properties of the SOBAM are then examined. We use an example to illustrate that the stability of the SOBAM is not guaranteed. For this result, we cannot use the conventional energy approach to estimate its memory capacity. Thus, we develop the statistical dynamics of the SOBAM. Given that a small number of errors appear in the initial input, the dynamics shows how the number of errors varies during recall. We use the dynamics to estimate the memory capacity, the attraction basin, and the number of errors in the retrieved items. Extension of the results to higher-order bidirectional associative memories is also discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • High-order and multilayer perceptron initialization

    Publication Year: 1997 , Page(s): 349 - 359
    Cited by:  Papers (36)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (300 KB)  

    Proper initialization is one of the most important prerequisites for fast convergence of feedforward neural networks like high-order and multilayer perceptrons. This publication aims at determining the optimal variance (or range) for the initial weights and biases, which is the principal parameter of random initialization methods for both types of neural networks. An overview of random weight initialization methods for multilayer perceptrons is presented. These methods are extensively tested using eight real-world benchmark data sets and a broad range of initial weight variances by means of more than 30000 simulations, in the aim to find the best weight initialization method for multilayer perceptrons. For high-order networks, a large number of experiments (more than 200000 simulations) was performed, using three weight distributions, three activation functions, several network orders, and the same eight data sets. The results of these experiments are compared to weight initialization techniques for multilayer perceptrons, which leads to the proposal of a suitable initialization method for high-order perceptrons. The conclusions on the initialization methods for both types of networks are justified by sufficiently small confidence intervals of the mean convergence times View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Using wavelet network in nonparametric estimation

    Publication Year: 1997 , Page(s): 227 - 236
    Cited by:  Papers (243)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (320 KB)  

    Wavelet networks are a class of neural networks consisting of wavelets. In this paper, algorithms for wavelet network construction are proposed for the purpose of nonparametric regression estimation. Particular attentions are paid to sparse training data so that problems of large dimension can be better handled. A numerical example on nonlinear system identification is presented for illustration View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neural-network approach to nonparametric and robust classification procedures

    Publication Year: 1997 , Page(s): 288 - 298
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (488 KB)  

    In this paper algorithms of neural-network type are introduced for solving estimation and classification problems when assumptions about independence, Gaussianity, and stationarity of the observation samples are no longer valid. Specifically, the asymptotic normality of several nonparametric classification tests is demonstrated and their implementation using a neural-network approach is presented. Initially, the neural nets train themselves via learning samples for nominal noise and alternative hypotheses distributions resulting in near optimum performance in a particular stochastic environment. In other than the nominal environments, however, high efficiency is maintained by adapting the optimum nonlinearities to changing conditions during operation via parallel networks, without disturbing the classification process. Furthermore, the superiority in performance of the proposed networks over more traditional neural nets is demonstrated in an application involving pattern recognition View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Capabilities of a four-layered feedforward neural network: four layers versus three

    Publication Year: 1997 , Page(s): 251 - 255
    Cited by:  Papers (65)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB)  

    Neural-network theorems state that only when there are infinitely many hidden units is a four-layered feedforward neural network equivalent to a three-layered feedforward neural network. In actual applications, however, the use of infinitely many hidden units is impractical. Therefore, studies should focus on the capabilities of a neural network with a finite number of hidden units, In this paper, a proof is given showing that a three-layered feedforward network with N-1 hidden units can give any N input-target relations exactly. Based on results of the proof, a four-layered network is constructed and is found to give any N input-target relations with a negligibly small error using only (N/2)+3 hidden units. This shows that a four-layered feedforward network is superior to a three-layered feedforward network in terms of the number of parameters needed for the training data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Unsupervised query-based learning of neural networks using selective-attention and self-regulation

    Publication Year: 1997 , Page(s): 205 - 217
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (308 KB)  

    Query-based learning (QBL) has been introduced for training a supervised network model with additional queried samples. Experiments demonstrated that the classification accuracy is further increased. Although QBL has been successfully applied to supervised neural networks, it is not suitable for unsupervised learning models without external supervisors. In this paper, an unsupervised QBL (UQBL) algorithm using selective-attention and self-regulation is proposed. Applying the selective-attention, we can ask the network to respond to its goal-directed behavior with self-focus. Since there is no supervisor to verify the self-focus, a compromise is then made to environment-focus with self-regulation. In this paper, we introduce UQBL1 and UQBL2 as two versions of UQBL; both of them can provide fast convergence. Our experiments indicate that the proposed methods are more insensitive to network initialization. They have better generalization performance and can be a significant reduction in their training size View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Discrete-time convergence theory and updating rules for neural networks with energy functions

    Publication Year: 1997 , Page(s): 445 - 447
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (96 KB)  

    We present convergence theorems for neural networks with arbitrary energy functions and discrete-time dynamics for both discrete and continuous neuronal input-output-functions. We discuss systematically how the neuronal updating rule should be extracted once an energy function is constructed for a given application, in order to guarantee the descent and minimization of the energy function as the network updates. We explain why the existing theory may lead to inaccurate results and oscillatory behaviors in the convergence process. We also point out the reason for and the side effects of using hysteresis neurons to suppress these oscillatory behaviors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Toward a general-purpose analog VLSI neural network with on-chip learning

    Publication Year: 1997 , Page(s): 413 - 423
    Cited by:  Papers (32)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB)  

    This paper describes elements necessary for a general-purpose low-cost very large scale integration (VLSI) neural network. By choosing a learning algorithm that is tolerant of analog nonidealities, the promise of high-density analog VLSI is realized. A 64-synapse, 8-neuron proof-of-concept chip is described. The synapse, which occupies only 4900 μm2 in a 2-μm technology, includes a hybrid of nonvolatile and dynamic weight storage that provides fast and accurate learning as well as reliable long-term storage with no refreshing. The architecture is user-configurable in any one-hidden-layer topology. The user-interface is fully microprocessor compatible. Learning is accomplished with minimal external support; the user need only present inputs, targets, and a clock. Learning is fast and reliable. The chip solves four-bit parity in an average of 680 ms and is successful in about 96% of the trials View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Temporal difference learning applied to sequential detection

    Publication Year: 1997 , Page(s): 278 - 287
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    This paper proposes a novel neural-network method for sequential detection, We first examine the optimal parametric sequential probability ratio test (SPRT) and make a simple equivalent transformation of the SPRT that makes it suitable for neural-network architectures. We then discuss how neural networks can learn the SPRT decision functions from observation data and labels. Conventional supervised learning algorithms have difficulties handling the variable length observation sequences, but a reinforcement learning algorithm, the temporal difference (TD) learning algorithm works ideally in training the neural network. The entire neural network is composed of context units followed by a feedforward neural network. The context units are necessary to store dynamic information that is needed to make good decisions. For an appropriate neural-network architecture, trained with independent and identically distributed (iid) observations by the TD learning algorithm, we show that the neural-network sequential detector can closely approximate the optimal SPRT with similar performance. The neural-network sequential detector has the additional advantage that it is a nonparametric detector that does not require probability density functions. Simulations demonstrated on iid Gaussian data show that the neural network and the SPRT have similar performance View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new recurrent neural-network architecture for visual pattern recognition

    Publication Year: 1997 , Page(s): 331 - 340
    Cited by:  Papers (16)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (324 KB)  

    We propose a new type of recurrent neural-network architecture, in which each output unit is connected to itself and is also fully connected to other output units and all hidden units. The proposed recurrent neural network differs from Jordan's and Elman's recurrent neural networks with respect to function and architecture, because it has been originally extended from being a mere multilayer feedforward neural network, to improve discrimination and generalization powers. We also prove the convergence properties of the learning algorithm in the proposed recurrent neural network, and analyze the performance of the proposed recurrent neural network by performing recognition experiments with the totally unconstrained handwritten numeric database of Concordia University, Montreal, Canada. Experimental results have confirmed that the proposed recurrent neural network improves discrimination and generalization powers in the recognition of visual patterns View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On neural networks that design neural associative memories

    Publication Year: 1997 , Page(s): 360 - 372
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (840 KB)  

    The design problem of generalized brain-state-in-a-box (GBSB) type associative memories is formulated as a constrained optimization program, and “designer” neural networks for solving the program in real time are proposed. The stability of the designer networks is analyzed using Barbalat's lemma. The analyzed and synthesized neural associative memories do not require symmetric weight matrices. Two types of the GBSB-based associative memories are analyzed, one when the network trajectories are constrained to reside in the hypercube [-1, 1]n and the other type when the network trajectories are confined to stay in the hypercube [0, 1]n. Numerical examples and simulations are presented to illustrate the results obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A simplification of the backpropagation-through-time algorithm for optimal neurocontrol

    Publication Year: 1997 , Page(s): 437 - 441
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (144 KB)  

    Backpropagation-through-time (BPTT) is the temporal extension of backpropagation which allows a multilayer neural network to approximate an optimal state-feedback control law provided some prior knowledge (Jacobian matrices) of the process is available. In this paper, a simplified version of the BPTT algorithm is proposed which more closely respects the principle of optimality of dynamic programming. Besides being simpler, the new algorithm is less time-consuming and allows in some cases the discovery of better control laws. A formal justification of this simplification is attempted by mixing the Lagrangian calculus underlying BPTT with Bellman-Hamilton-Jacobi equations. The improvements due to this simplification are illustrated by two optimal control problems: the rendezvous and the bioreactor View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Growing a hypercubical output space in a self-organizing feature map

    Publication Year: 1997 , Page(s): 218 - 226
    Cited by:  Papers (31)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    Neural maps project data from an input space onto a neuron position in a (often lower dimensional) output space grid in a neighborhood preserving way, with neighboring neurons in the output space responding to neighboring data points in the input space. A map-learning algorithm can achieve an optimal neighborhood preservation only, if the output space topology roughly matches the effective structure of the data in the input space. We here present a growth algorithm, called the GSOM or growing self-organizing map, which enhances a widespread map self-organization process, Kohonen's self-organizing feature map (SOFM), by an adaptation of the output space grid during learning. The GSOM restricts the output space structure to the shape of a general hypercubical shape, with the overall dimensionality of the grid and its extensions along the different directions being subject of the adaptation. This constraint meets the demands of many larger information processing systems, of which the neural map can be a part. We apply our GSOM-algorithm to three examples, two of which involve real world data. Using recently developed methods for measuring the degree of neighborhood preservation in neural maps, we find the GSOM-algorithm to produce maps which preserve neighborhoods in a nearly optimal fashion View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Mapping and hierarchical self-organizing neural networks for VLSI placement

    Publication Year: 1997 , Page(s): 299 - 314
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (668 KB)  

    We have developed mapping and hierarchical self-organizing neural networks for placement of very large scale integrated (VLST) circuits. In this paper, we introduce MHSO and MHSO2 as two versions of mapping and hierarchical self-organizing network (MHSO) algorithms. By using the MHSO, each module in the placement wins the competition with a probability density function that is defined according to different design styles, e.g., the gate arrays and standard cell circuits. The relation between a placement carrier and movable modules is met by the algorithm's ability to map an input space (somatosensory source) into an output space where the circuit modules are located, MHSO2 is designed for macro cell circuits. In this algorithm, the shape and dimension of each module is simultaneously considered together with the wire length by a hierarchical order. In comparison with other conventional placement approaches, the MHSO algorithms have shown their distinct advantages. The results for benchmark circuits so far obtained are quite comparable to simulated annealing (SA), but the computation time is about eight-ten times faster than with SA View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust speech recognition based on joint model and feature space optimization of hidden Markov models

    Publication Year: 1997 , Page(s): 194 - 204
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    The hidden Markov model (HMM) inversion algorithm, based on either the gradient search or the Baum-Welch reestimation of input speech features, is proposed and applied to the robust speech recognition tasks under general types of mismatch conditions. This algorithm stems from the gradient-based inversion algorithm of an artificial neural network (ANN) by viewing an HMM as a special type of ANN. Given input speech features s, the forward training of an HMM finds the model parameters λ subject to an optimization criterion. On the other hand, the inversion of an HMM finds speech features, s, subject to an optimization criterion with given model parameters λ. The gradient-based HMM inversion and the Baum-Welch HMM inversion algorithms can be successfully integrated with the model space optimization techniques, such as the robust MINIMAX technique, to compensate the mismatch in the joint model and feature space. The joint space mismatch compensation technique achieves better performance than the single space, i.e. either the model space or the feature space alone, mismatch compensation techniques. It is also demonstrated that approximately 10-dB signal-to-noise ratio (SNR) gain is obtained in the low SNR environments when the joint model and feature space mismatch compensation technique is used View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Artificial neural networks controlled fast valving in a power generation plant

    Publication Year: 1997 , Page(s): 373 - 389
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (504 KB)  

    This paper presents an artificial neural-network-based controller to realize the fast valving in a power generation plant. The backpropagation algorithm is used to train the feedforward neural networks controller. The hardware implementation and the test results of the controller on a physical pilot-scale power plant setup are described in detail. Compared with the conventional fast valving methods applied to the same system, test results both with the computer simulation and on a physical pilot-scale power plant setup demonstrate that the artificial neural network controller has satisfactory generalization capability, reliability, and accuracy to be feasible for this critical control operation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Modified self-organizing feature map algorithms for efficient digital hardware implementation

    Publication Year: 1997 , Page(s): 315 - 330
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (524 KB)  

    This paper describes two variants of the Kohonen's self-organizing feature map (SOFM) algorithm. Both variants update the weights only after presentation of a group of input vectors. In contrast, in the original algorithm the weights are updated after presentation of every input vector. The main advantage of these variants is to make available a finer grain of parallelism, for implementation on machines with a very large number of processors, without compromising the desired properties of the algorithm. In this work it is proved that, for one-dimensional (1-D) maps and 1-D continuous input and weight spaces, the strictly increasing or decreasing weight configuration forms an absorbing class in both variants, exactly as in the original algorithm. Ordering of the maps and convergence to asymptotic values are also proved, again confirming the theoretical results obtained for the original algorithm. Simulations of a real-world application using two-dimensional (2-D) maps on 12-D speech data are presented to back up the theoretical results and show that the performance of one of the variants is in all respects almost as good as the original algorithm. Finally, the practical utility of the finer parallelism made available is confirmed by the description of a massively parallel hardware system that makes effective use of the best variant View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope