By Topic

Neural Networks, IEEE Transactions on

Issue 2 • Date March 1997

Filter Results

Displaying Results 1 - 25 of 30
  • On the "Identification and control of dynamical systems using neural networks"

    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    Referring to the above said paper by Narendra-Parthasarathy (ibid., vol.1, p4-27 (1990)), it is noted that the given Example 2 (p.15) has a third equilibrium state corresponding to the point (0.5, 0.5). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cascade ARTMAP: integrating neural computation and symbolic knowledge processing

    Page(s): 237 - 250
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (383 KB)  

    This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Comments on "Stochastic choice of basis functions in adaptive function approximation and the functional-link net" [with reply]

    Page(s): 452 - 454
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (52 KB)  

    This paper includes some comments and amendments of the above-mentioned paper by Igelnik et al. (1995). Subsequently, Theorem 1 in the above-mentioned paper has been revised. The significant change of the original theorem is the space of the thresholds in the hidden layer. The revised theorem says that the thresholds of hidden b/sub 0/, should be -w/sub 0//spl middot/y/sub 0/-u/sub 0/, where w/sub 0/=/spl alpha/w/spl circ//sub 0/; w/spl circ//sub 0/=(w/spl circ//sub 01/, /spl middot//spl middot//spl middot/, y/sub 0d/), and u/sub 0/ be independent and uniformly distributed in V/sup d/=[0; /spl Omega/]/spl times/[-/spl Omega/; /spl Omega/]/sup d-1/, I/sup d/, and [-2d/spl Omega/, 2d/spl Omega/], respectively. In reply, Igelnik et al. acknowledge that a factor of two was omitted in the statement of a trigonometric identity. However, the validity of the essential point of Theorem 1 is unaltered. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recent Advances in Reinforcement Learning [Books in Brief]

    Page(s): 456
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • Bayesian Learning for Neural Networks [Books in Brief]

    Page(s): 456
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • Topology preservation in self-organizing feature maps: exact definition and measurement

    Page(s): 256 - 266
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (556 KB)  

    The neighborhood preservation of self-organizing feature maps like the Kohonen map is an important property which is exploited in many applications. However, if a dimensional conflict arises this property is lost. Various qualitative and quantitative approaches are known for measuring the degree of topology preservation. They are based on using the locations of the synaptic weight vectors. These approaches, however, may fail in case of nonlinear data manifolds. To overcome this problem, in this paper we present an approach which uses what we call the induced receptive fields for determining the degree of topology preservation. We first introduce a precise definition of topology preservation and then propose a tool for measuring it, the topographic function. The topographic function vanishes if and only if the map is topology preserving. We demonstrate the power of this tool for various examples of data manifolds View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust speech recognition based on joint model and feature space optimization of hidden Markov models

    Page(s): 194 - 204
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    The hidden Markov model (HMM) inversion algorithm, based on either the gradient search or the Baum-Welch reestimation of input speech features, is proposed and applied to the robust speech recognition tasks under general types of mismatch conditions. This algorithm stems from the gradient-based inversion algorithm of an artificial neural network (ANN) by viewing an HMM as a special type of ANN. Given input speech features s, the forward training of an HMM finds the model parameters λ subject to an optimization criterion. On the other hand, the inversion of an HMM finds speech features, s, subject to an optimization criterion with given model parameters λ. The gradient-based HMM inversion and the Baum-Welch HMM inversion algorithms can be successfully integrated with the model space optimization techniques, such as the robust MINIMAX technique, to compensate the mismatch in the joint model and feature space. The joint space mismatch compensation technique achieves better performance than the single space, i.e. either the model space or the feature space alone, mismatch compensation techniques. It is also demonstrated that approximately 10-dB signal-to-noise ratio (SNR) gain is obtained in the low SNR environments when the joint model and feature space mismatch compensation technique is used View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Unsupervised query-based learning of neural networks using selective-attention and self-regulation

    Page(s): 205 - 217
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (308 KB)  

    Query-based learning (QBL) has been introduced for training a supervised network model with additional queried samples. Experiments demonstrated that the classification accuracy is further increased. Although QBL has been successfully applied to supervised neural networks, it is not suitable for unsupervised learning models without external supervisors. In this paper, an unsupervised QBL (UQBL) algorithm using selective-attention and self-regulation is proposed. Applying the selective-attention, we can ask the network to respond to its goal-directed behavior with self-focus. Since there is no supervisor to verify the self-focus, a compromise is then made to environment-focus with self-regulation. In this paper, we introduce UQBL1 and UQBL2 as two versions of UQBL; both of them can provide fast convergence. Our experiments indicate that the proposed methods are more insensitive to network initialization. They have better generalization performance and can be a significant reduction in their training size View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A binary Hopfield neural-network approach for satellite broadcast scheduling problems

    Page(s): 441 - 445
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (144 KB)  

    This paper presents a binary Hopfield neural network approach for finding a broadcasting schedule in a low-altitude satellite system. Our neural network is composed of simple binary neurons on the synchronous parallel computation, which is greatly suitable for implementation on a digital machine. With the help of heuristic methods, the neural network of a maximum of 200000 neurons can always find near-optimum solutions on a conventional workstation in our simulations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neural-network approach to nonparametric and robust classification procedures

    Page(s): 288 - 298
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (488 KB)  

    In this paper algorithms of neural-network type are introduced for solving estimation and classification problems when assumptions about independence, Gaussianity, and stationarity of the observation samples are no longer valid. Specifically, the asymptotic normality of several nonparametric classification tests is demonstrated and their implementation using a neural-network approach is presented. Initially, the neural nets train themselves via learning samples for nominal noise and alternative hypotheses distributions resulting in near optimum performance in a particular stochastic environment. In other than the nominal environments, however, high efficiency is maintained by adapting the optimum nonlinearities to changing conditions during operation via parallel networks, without disturbing the classification process. Furthermore, the superiority in performance of the proposed networks over more traditional neural nets is demonstrated in an application involving pattern recognition View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Growing a hypercubical output space in a self-organizing feature map

    Page(s): 218 - 226
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    Neural maps project data from an input space onto a neuron position in a (often lower dimensional) output space grid in a neighborhood preserving way, with neighboring neurons in the output space responding to neighboring data points in the input space. A map-learning algorithm can achieve an optimal neighborhood preservation only, if the output space topology roughly matches the effective structure of the data in the input space. We here present a growth algorithm, called the GSOM or growing self-organizing map, which enhances a widespread map self-organization process, Kohonen's self-organizing feature map (SOFM), by an adaptation of the output space grid during learning. The GSOM restricts the output space structure to the shape of a general hypercubical shape, with the overall dimensionality of the grid and its extensions along the different directions being subject of the adaptation. This constraint meets the demands of many larger information processing systems, of which the neural map can be a part. We apply our GSOM-algorithm to three examples, two of which involve real world data. Using recently developed methods for measuring the degree of neighborhood preservation in neural maps, we find the GSOM-algorithm to produce maps which preserve neighborhoods in a nearly optimal fashion View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Convergence under dynamical thresholds with delays

    Page(s): 341 - 348
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    Necessary and sufficient conditions are obtained for the existence of a globally asymptotically stable equilibrium of a class of delay differential equations modeling the action of a neuron with dynamical threshold effects View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Artificial neural networks controlled fast valving in a power generation plant

    Page(s): 373 - 389
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (504 KB)  

    This paper presents an artificial neural-network-based controller to realize the fast valving in a power generation plant. The backpropagation algorithm is used to train the feedforward neural networks controller. The hardware implementation and the test results of the controller on a physical pilot-scale power plant setup are described in detail. Compared with the conventional fast valving methods applied to the same system, test results both with the computer simulation and on a physical pilot-scale power plant setup demonstrate that the artificial neural network controller has satisfactory generalization capability, reliability, and accuracy to be feasible for this critical control operation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural-network front ends in unsupervised learning

    Page(s): 390 - 401
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2004 KB)  

    Proposed is an idea of partial supervision realized in the form of a neural-network front end to the schemes of unsupervised learning (clustering). This neural network leads to an anisotropic nature of the induced feature space. The anisotropic property of the space provides us with some of its local deformation necessary to properly represent labeled data and enhance efficiency of the mechanisms of clustering to be exploited afterwards. The training of the network is completed based upon available labeled patterns-a referential form of the labeling gives rise to reinforcement learning. It is shown that the discussed approach is universal and can be utilized in conjunction with any clustering method. Experimental studies are concentrated on three main categories of unsupervised learning including FUZZY ISODATA, Kohonen self-organizing maps, and hierarchical clustering View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Temporal difference learning applied to sequential detection

    Page(s): 278 - 287
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    This paper proposes a novel neural-network method for sequential detection, We first examine the optimal parametric sequential probability ratio test (SPRT) and make a simple equivalent transformation of the SPRT that makes it suitable for neural-network architectures. We then discuss how neural networks can learn the SPRT decision functions from observation data and labels. Conventional supervised learning algorithms have difficulties handling the variable length observation sequences, but a reinforcement learning algorithm, the temporal difference (TD) learning algorithm works ideally in training the neural network. The entire neural network is composed of context units followed by a feedforward neural network. The context units are necessary to store dynamic information that is needed to make good decisions. For an appropriate neural-network architecture, trained with independent and identically distributed (iid) observations by the TD learning algorithm, we show that the neural-network sequential detector can closely approximate the optimal SPRT with similar performance. The neural-network sequential detector has the additional advantage that it is a nonparametric detector that does not require probability density functions. Simulations demonstrated on iid Gaussian data show that the neural network and the SPRT have similar performance View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A simplification of the backpropagation-through-time algorithm for optimal neurocontrol

    Page(s): 437 - 441
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (144 KB)  

    Backpropagation-through-time (BPTT) is the temporal extension of backpropagation which allows a multilayer neural network to approximate an optimal state-feedback control law provided some prior knowledge (Jacobian matrices) of the process is available. In this paper, a simplified version of the BPTT algorithm is proposed which more closely respects the principle of optimality of dynamic programming. Besides being simpler, the new algorithm is less time-consuming and allows in some cases the discovery of better control laws. A formal justification of this simplification is attempted by mixing the Lagrangian calculus underlying BPTT with Bellman-Hamilton-Jacobi equations. The improvements due to this simplification are illustrated by two optimal control problems: the rendezvous and the bioreactor View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • How initial conditions affect generalization performance in large networks

    Page(s): 448 - 451
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (120 KB)  

    Generalization is one of the most important problems in neural-network research. It is influenced by several factors in the network design, such as network size, weight decay factor, and others. We show here that the initial weight distribution (for gradient decent training algorithms) is one other factor that influences generalization. The initial conditions guide the training algorithm to search particular places of the weight space. For instance small initial weights tend to result in low complexity networks, and therefore can effectively act as a regularization factor. We propose a novel network complexity measure, which is helpful in shedding insight into the phenomenon, as well as in studying other aspects of generalization View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Iterative generation of higher-order nets in polynomial time using linear programming

    Page(s): 402 - 412
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    This paper presents an algorithm for constructing and training a class of higher-order perceptrons for classification problems. The method uses linear programming models to construct and train the net. Its polynomial time complexity is proven and computational results are provided for several well-known problems. In all cases, very small nets were created compared to those reported in other computational studies View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new recurrent neural-network architecture for visual pattern recognition

    Page(s): 331 - 340
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (324 KB)  

    We propose a new type of recurrent neural-network architecture, in which each output unit is connected to itself and is also fully connected to other output units and all hidden units. The proposed recurrent neural network differs from Jordan's and Elman's recurrent neural networks with respect to function and architecture, because it has been originally extended from being a mere multilayer feedforward neural network, to improve discrimination and generalization powers. We also prove the convergence properties of the learning algorithm in the proposed recurrent neural network, and analyze the performance of the proposed recurrent neural network by performing recognition experiments with the totally unconstrained handwritten numeric database of Concordia University, Montreal, Canada. Experimental results have confirmed that the proposed recurrent neural network improves discrimination and generalization powers in the recognition of visual patterns View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Capabilities of a four-layered feedforward neural network: four layers versus three

    Page(s): 251 - 255
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB)  

    Neural-network theorems state that only when there are infinitely many hidden units is a four-layered feedforward neural network equivalent to a three-layered feedforward neural network. In actual applications, however, the use of infinitely many hidden units is impractical. Therefore, studies should focus on the capabilities of a neural network with a finite number of hidden units, In this paper, a proof is given showing that a three-layered feedforward network with N-1 hidden units can give any N input-target relations exactly. Based on results of the proof, a four-layered network is constructed and is found to give any N input-target relations with a negligibly small error using only (N/2)+3 hidden units. This shows that a four-layered feedforward network is superior to a three-layered feedforward network in terms of the number of parameters needed for the training data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability and statistical properties of second-order bidirectional associative memory

    Page(s): 267 - 277
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (536 KB)  

    In this paper, a bidirectional associative memory (BAM) model with second-order connections, namely second-order bidirectional associative memory (SOBAM), is first reviewed. The stability and statistical properties of the SOBAM are then examined. We use an example to illustrate that the stability of the SOBAM is not guaranteed. For this result, we cannot use the conventional energy approach to estimate its memory capacity. Thus, we develop the statistical dynamics of the SOBAM. Given that a small number of errors appear in the initial input, the dynamics shows how the number of errors varies during recall. We use the dynamics to estimate the memory capacity, the attraction basin, and the number of errors in the retrieved items. Extension of the results to higher-order bidirectional associative memories is also discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Mapping and hierarchical self-organizing neural networks for VLSI placement

    Page(s): 299 - 314
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (668 KB)  

    We have developed mapping and hierarchical self-organizing neural networks for placement of very large scale integrated (VLST) circuits. In this paper, we introduce MHSO and MHSO2 as two versions of mapping and hierarchical self-organizing network (MHSO) algorithms. By using the MHSO, each module in the placement wins the competition with a probability density function that is defined according to different design styles, e.g., the gate arrays and standard cell circuits. The relation between a placement carrier and movable modules is met by the algorithm's ability to map an input space (somatosensory source) into an output space where the circuit modules are located, MHSO2 is designed for macro cell circuits. In this algorithm, the shape and dimension of each module is simultaneously considered together with the wire length by a hierarchical order. In comparison with other conventional placement approaches, the MHSO algorithms have shown their distinct advantages. The results for benchmark circuits so far obtained are quite comparable to simulated annealing (SA), but the computation time is about eight-ten times faster than with SA View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Using wavelet network in nonparametric estimation

    Page(s): 227 - 236
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (320 KB)  

    Wavelet networks are a class of neural networks consisting of wavelets. In this paper, algorithms for wavelet network construction are proposed for the purpose of nonparametric regression estimation. Particular attentions are paid to sparse training data so that problems of large dimension can be better handled. A numerical example on nonlinear system identification is presented for illustration View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A parallel processing VLSI BAM engine

    Page(s): 424 - 436
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (748 KB)  

    In this paper emerging parallel/distributed architectures are explored for the digital VLSI implementation of adaptive bidirectional associative memory (BAM) neural network. A single instruction stream many data stream (SIMD)-based parallel processing architecture, is developed for the adaptive BAM neural network, taking advantage of the inherent parallelism in BAM. This novel neural processor architecture is named the sliding feeder BAM array processor (SLiFBAM). The SLiFBAM processor can be viewed as a two-stroke neural processing engine, It has four operating modes: learn pattern, evaluate pattern, read weight, and write weight. Design of a SLiFBAM VLSI processor chip is also described. By using 2-μm scalable CMOS technology, a SLiFBAM processor chip with 4+4 neurons and eight modules of 256×5 bit local weight-storage SRAM, was integrated on a 6.9×7.4 mm2 prototype die. The system architecture is highly flexible and modular, enabling the construction of larger BAM networks of up to 252 neurons using multiple SLiFBAM chips View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope