By Topic

Neural Networks, IEEE Transactions on

Issue 5 • Date Sept. 1997

Filter Results

Displaying Results 1 - 25 of 30
  • Comments on "A training rule which guarantees finite-region stability for a class of closed-loop neural-network control systems" [with reply]

    Page(s): 1217 - 1218
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (30 KB)  

    In the above paper by Kuntanapreeda-Fullmer (ibid., vol.7, no.3 (1996)) a training method for a neural-network control system which guarantees local closed-loop stability is proposed based on a Lyapunov function and a modified standard backpropagation training rule. In this letter, we show that the proof of Proposition 1 and the proposed stability condition as training constraints are not correct and therefore that the stability of the neural-network control system is not quite right. We suggest a modified version of the proposition with its proof and comment on another problem of the paper. In reply, Kuntanapreeda-Fullmer maintain the proof in the original paper is correct. Rather than identifying an error, they believe Park et al. have made a significant extension of the proof for application to stable online training networks. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author's reply

    Page(s): 1218
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (10 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Foundations Of Neural Networks, Fuzzy Systems, And Knowledge Engineering [Books in Brief]

    Page(s): 1219
    Save to Project icon | Request Permissions | PDF file iconPDF (9 KB)  
    Freely Available from IEEE
  • Neuro-fuzzy And Soft Computing: A Computational Approach To Learning And Machine Intelligence [Books in Brief]

    Page(s): 1219
    Save to Project icon | Request Permissions | PDF file iconPDF (9 KB)  
    Freely Available from IEEE
  • An ART1 microchip and its use in multi-ART1 systems

    Page(s): 1184 - 1194
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (508 KB)  

    Recently, a real-time clustering microchip neural engine based on the ART1 architecture has been reported. However, that chip rendered an extremely high silicon area consumption of 1 cm2, and consequently an extremely low yield of 6%. Redundant circuit techniques can be introduced to improve yield performance at the cost of further increasing chip size. In this paper we present an improved ART1 chip prototype based on a different approach to implement the most area consuming circuit elements of the first prototype: an array of several thousand current sources which have to match within a precision of around 1%. Such achievement was possible after a careful transistor mismatch characterization of the fabrication process (ES2-1.0 μm CMOS). A new prototype chip has been fabricated which can cluster 50-b input patterns into up to ten categories. The chip has 15 times less area, shows a yield performance of 98%, and presents the same precision and speed than the previous prototype. Due to its higher robustness multichip systems are easily assembled. As a demonstration we show results of a two-chip ART1 system, and of an ARTMAP system made of two ART1 chips and an extra interfacing chip View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Objective functions for training new hidden units in constructive neural networks

    Page(s): 1131 - 1148
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (676 KB)  

    In this paper, we study a number of objective functions for training new hidden units in constructive algorithms for multilayer feedforward networks. The aim is to derive a class of objective functions the computation of which and the corresponding weight updates can be done in O(N) time, where N is the number of training patterns. Moreover, even though input weight freezing is applied during the process for computational efficiency, the convergence property of the constructive algorithms using these objective functions is still preserved. We also propose a few computational tricks that can be used to improve the optimization of the objective functions under practical situations. Their relative performance in a set of two-dimensional regression problems is also discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • VLSI circuit placement with rectilinear modules using three-layer force-directed self-organizing maps

    Page(s): 1049 - 1064
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (368 KB)  

    In this paper, a three-layer force-directed self-organizing map is designed to resolve the circuit placement problem with arbitrarily shaped rectilinear modules. The proposed neural model with an additional hidden layer can easily model a rectilinear module by a set of hidden neurons to correspond the partitioned rectangles. With the collective computing from hidden neurons, these rectilinear modules can correctly interact with each other and finally converge to a good placement result. In this paper, multiple contradictory criteria are accounted simultaneously during the placement process, in which, both the wire length and the module overlap are reduced. The proposed model has been successfully exploited to solve the time consuming rectilinear module placement problem. The placement results of real rectilinear test examples are presented, which demonstrate that the proposed method is better than the simulated annealing approach in the total wire length. The appropriate parameter values which yield good solutions are also investigated View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic statistical theory of overtraining and cross-validation

    Page(s): 985 - 996
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (516 KB)  

    A statistical theory for overtraining is proposed. The analysis treats general realizable stochastic neural networks, trained with Kullback-Leibler divergence in the asymptotic case of a large number of training examples. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Based on the cross-validation stopping we consider the ratio the examples should be divided into training and cross-validation sets in order to obtain the optimum performance. Although cross-validated early stopping is useless in the asymptotic region, it surely decreases the generalization error in the nonasymptotic region. Our large scale simulations done on a CM5 are in good agreement with our analytical findings View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Glove-talk II - a neural-network interface which maps gestures to parallel formant speech synthesizer controls

    Page(s): 977 - 984
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (152 KB)  

    Glove-Talk II is a system which translates hand gestures to speech through an adaptive interface. Hand gestures are mapped continuously to ten control parameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-Talk II uses several input devices, a parallel formant speech synthesizer, and three neural networks. The gesture-to-speech task is divided into vowel and consonant production by using a gating network to weight the outputs of a vowel and a consonant neural network. The gating network and the consonant network are trained with examples from the user. The vowel network implements a fixed user-defined relationship between hand position and vowel sound and does not require any training examples from the user. Volume, fundamental frequency, and stop consonants are produced with a fixed mapping from the input devices. With Glove-Talk II, the subject can speak slowly but with far more natural sounding pitch variations than a text-to-speech synthesizer View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Morphological shared-weight networks with applications to automatic target recognition

    Page(s): 1195 - 1203
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    A shared-weight neural network based on mathematical morphology is introduced. The feature extraction process is learned by interaction with the classification process. Feature extraction is performed using gray-scale hit-miss transforms that are independent of gray-level shifts. The morphological shared-weight neural network (MSNN) is applied to automatic target recognition. Two sets of images of outdoor scenes are considered. The first set consists of two subsets of infrared images of tracked vehicles. The goal in this set is to reject the background and to detect tracked vehicles. The second set consists of visible images of cars in a parking lot. The goal in this set is to detect the Chevrolet Blazers with various degrees of occlusion. A training method that is effective in reducing false alarms and a target aim point selection algorithm are introduced. The MSNN is compared to the standard shared-weight neural network. The MSNN trains relatively quickly and exhibits better generalization View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive critic designs

    Page(s): 997 - 1007
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (344 KB)  

    We discuss a variety of adaptive critic designs (ACDs) for neurocontrol. These are suitable for learning in noisy, nonlinear, and nonstationary environments. They have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Our discussion of these origins leads to an explanation of three design families: heuristic dynamic programming, dual heuristic programming, and globalized dual heuristic programming (GDHP). The main emphasis is on DHP and GDHP as advanced ACDs. We suggest two new modifications of the original GDHP design that are currently the only working implementations of GDHP. They promise to be useful for many engineering applications in the areas of optimization and optimal control. Based on one of these modifications, we present a unified approach to all ACDs. This leads to a generalized training procedure for ACDs View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A minor subspace analysis algorithm

    Page(s): 1149 - 1155
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    This paper proposes a learning algorithm which extracts adaptively the minor subspace spanned by the eigenvectors corresponding to the smallest eigenvalues of the autocorrelation matrix of an input signal. We show both analytically and by simulation results that the weight vectors provided by the proposed algorithm are guaranteed to converge to the minor subspace of the input signal. For wider applications, we also present the complex valued version of the proposed minor subspace analysis algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Time-delay neural networks: representation and induction of finite-state machines

    Page(s): 1065 - 1070
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (156 KB)  

    In this work, we characterize and contrast the capabilities of the general class of time-delay neural networks (TDNNs) with input delay neural networks (IDNNs), the subclass of TDNNs with delays limited to the inputs. Each class of networks is capable of representing the same set of languages, those embodied by the definite memory machines (DMMs), a subclass of finite-state machines. We demonstrate the close affinity between TDNNs and DMM languages by learning a very large DMM (2048 states) using only a few training examples. Even though both architectures are capable of representing the same class of languages, they have distinguishable learning biases. Intuition suggests that general TDNNs which include delays in hidden layers should perform well, compared to IDNNs, on problems in which the output can be expressed as a function on narrow input windows which repeat in time. On the other hand, these general TDNNs should perform poorly when the input windows are wide, or there is little repetition. We confirm these hypotheses via a set of simulations and statistical analysis View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning and adaptation in an airborne laser fire controller

    Page(s): 1078 - 1089
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    A simulated battlefield, containing airborne lasers that shoot ballistic missiles down, provides an excellent test-bed for developing adaptive controllers. An airborne laser fire controller, which can adapt the strategy it uses for target selection, is developed. The approach is to transform a knowledge-based controller into an adaptable connectionist representation, use supervised training to initialize the weights so that the adaptable controller mimics the knowledge-based controller, and then use directed search with simulation-based performance evaluation to continuously adapt the controller behavior to the dynamic environmental conditions. New knowledge can be directly extracted from the automatically discovered controllers. Three directed search methods are characterized for production training, and compared with the better characterized gradient descent methods commonly used for supervised training. Automated discovery of improved controllers is demonstrated, as is automated adaptation of controller behavior to changes in environmental conditions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A theoretical study of linear and nonlinear equalization in nonlinear magnetic storage channels

    Page(s): 1106 - 1118
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (412 KB)  

    We present methods to systematically design a feedforward neural-network detector from the knowledge of the channel characteristics. Its performance is compared with the conventional linear equalizer in a magnetic recording channel suffering from signal-dependent noise and nonlinear intersymbol interference. The superiority of the nonlinear schemes are clearly observed in all cases studied, especially in the presence of severe nonlinearity and noise. We also show that the decision boundaries formed by a theoretically derived neural-network classifier are geometrically close to those of a neural network trained by the backpropagation algorithm. The approach in this work is suitable for quantifying the gain in using a neural-network method as opposed to linear methods in the classification of noisy patterns View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Yet another algorithm which can generate topography map

    Page(s): 1204 - 1207
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (168 KB)  

    This paper presents an algorithm to form a topographic map resembling to the self-organizing map. The idea stems on defining an energy function which reveals the local correlation between neighboring neurons. The larger the value of the energy function, the higher the correlation of the neighborhood neurons. On this account, the proposed algorithm is defined as the gradient ascent of this energy function. Simulations on two-dimensional maps are illustrated View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Vector-entropy optimization-based neural-network approach to image reconstruction from projections

    Page(s): 1008 - 1014
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    In this paper we propose a multiobjective decision making based neural-network model and algorithm for image reconstruction from projections. This model combines the Hopfield's model and multiobjective decision making approach. We develop a weighted sum optimization based neural-network algorithm. The dynamical process of the net is based on minimization of a weighted sum energy function and Euler's iteration, and apply this algorithm to image reconstruction from computer-generated noisy projections and Siemens Somatson DR scanner data, respectively. Reconstructions based on this method is shown to be superior to conventional iterative reconstruction algorithms such as the multiplicate algebraic reconstruction technique (MART) and convolution from the point of view of accuracy of reconstruction. Computer simulation using the multiobjective method shows a significant improvement in image quality and convergence behavior over the conventional algorithms View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Diffusion approximation of frequency sensitive competitive learning

    Page(s): 1026 - 1030
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (188 KB)  

    The focus of this paper is a convergence study of the frequency sensitive competitive learning (FSCL) algorithm. We approximate the final phase of FSCL learning by a diffusion process described by the Fokker-Plank equation. Sufficient and necessary conditions are presented for the convergence of the diffusion process to a local equilibrium. The analysis parallels that by Ritter-Schulten (1988) for Kohonen's self-organizing map. We show that the convergence conditions involve only the learning rate and that they are the same as the conditions for weak convergence described previously. Our analysis thus broadens the class of algorithms that have been shown to have these types of convergence characteristics View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning rules for neuro-controller via simultaneous perturbation

    Page(s): 1119 - 1130
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (276 KB)  

    This paper describes learning rules using simultaneous perturbation for a neurocontroller that controls an unknown plant. When we apply a direct control scheme by a neural network, the neural network must learn an inverse system of the unknown plant. In this case, we must know the sensitivity function of the plant using a kind of the gradient method as a learning rule of the neural network. On the other hand, the learning rules described here do not require information about the sensitivity function. Some numerical simulations of a two-link planar arm and a tracking problem for a nonlinear dynamic plant are shown View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On competitive learning

    Page(s): 1214 - 1217
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (128 KB)  

    We derive learning rates such that all training patterns are equally important statistically and the learning outcome is independent of the order in which training patterns are presented, if the competitive neurons win the same sets of training patterns regardless the order of presentation. We show that under these schemes, the learning rules in the two different weight normalization approaches, the length-constraint and the sum-constraint, yield practically the same results, if the competitive neurons win the same sets of training patterns with both constraints. These theoretical results are illustrated with computer simulations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Are artificial neural networks black boxes?

    Page(s): 1156 - 1164
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (260 KB)  

    Artificial neural networks are efficient computing models which have shown their strengths in solving hard problems in artificial intelligence. They have also been shown to be universal approximators. Notwithstanding, one of the major criticisms is their being black boxes, since no satisfactory explanation of their behavior has been offered. In this paper, we provide such an interpretation of neural networks so that they will no longer be seen as black boxes. This is stated after establishing the equality between a certain class of neural nets and fuzzy rule-based systems. This interpretation is built with fuzzy rules using a new fuzzy logic operator which is defined after introducing the concept of f-duality. In addition, this interpretation offers an automated knowledge acquisition procedure View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the use of artificial neural networks for the analysis of survival data

    Page(s): 1071 - 1077
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (124 KB)  

    Artificial neural networks are a powerful tool for analyzing data sets where there are complicated nonlinear interactions between the measured inputs and the quantity to be predicted. We show that the results obtained when neural networks are applied to survival data depend critically on the treatment of censoring in the data. When the censoring is modeled correctly, neural networks are a robust model independent technique for the analysis of very large sets of survival data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An evaluation of the neocognitron

    Page(s): 1090 - 1105
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (540 KB)  

    We describe a sequence of experiments investigating the strengths and limitations of Fukushima's neocognitron as a handwritten digit classifier. Using the results of these experiments as a foundation, we propose and evaluate improvements to Fukushima's original network in an effort to obtain higher recognition performance. The neocognitron performance is shown to be strongly dependent on the choice of selectivity parameters and we present two methods to adjust these variables. Performance of the network under the more effective of the two new selectivity adjustment techniques suggests that the network fails to exploit the features that distinguish different classes of input data. To avoid this shortcoming, the network's final layer cells were replaced by a nonlinear classifier (a multilayer perceptron) to create a hybrid architecture. Tests of Fukushima's original system and the novel systems proposed in this paper suggest that it may be difficult for the neocognitron to achieve the performance of existing digit classifiers due to its reliance upon the supervisor's choice of selectivity parameters and training data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the convergence rate performance of the normalized least-mean-square adaptation

    Page(s): 1211 - 1214
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (144 KB)  

    This paper compares the convergence rate performance of the normalized least-mean-square (NLMS) algorithm to that of the standard least-mean-square (LMS) algorithm, which is based on a well-known interpretation of the NLMS algorithm as a form of the LMS via input normalization. With this interpretation, the analysis is considerably simplified and the difference in rate of parameter convergence can be compared directly by evaluating both the condition number of the normalized and unnormalized input correlation matrix. This paper derives the condition number expressions for the normalized input correlation matrix of which the arbitrary-length filter model is linear with respect to its adaptable parameters and contain only two distinct unnormalized eigenvalues. These expressions, which require that the input samples be statistically stationary and zero-mean Gaussian distributed, provide an important insight into the relative convergence performance of the NLMS algorithm to that of the LMS as a function of filter length. This paper also provides a conjecture which set bounds on the NLMS condition number for any arbitrary number of distinct unnormalized eigenvalues View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Kernel orthonormalization in radial basis function neural networks

    Page(s): 1177 - 1183
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    This paper deals with optimization of the computations involved in training radial basis function (RBF) neural networks. The main contribution of the reported work is the method for network weights calculation, in which the key idea is to transform the RBF kernels into an orthonormal set of functions (using the standard Gram-Schmidt orthogonalization). This significantly reduces the computing time if the RBF training scheme, which relies on adding one kernel hidden node at a time to improve network performance, is adopted. Another property of the method is that, after the RBF network weights are computed, the original network structure can be restored back. An additional strength of the method is the possibility to decompose the proposed computing task into a number of parallel subtasks so gaining further savings on computing time. Also, the proposed weight calculation technique has low storage requirements. These features make the method very attractive for hardware implementation. The paper presents a detailed derivation of the proposed network weights calculation procedure and demonstrates its validity for RBF network training on a number of data classification and function approximation problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope