By Topic

Neural Networks, IEEE Transactions on

Issue 3 • Date May 1997

Filter Results

Displaying Results 1 - 25 of 37
  • Comments on "Diagonal recurrent neural networks for dynamic systems control". Reproof of theorems 2 and 4 [with reply]

    Publication Year: 1997 , Page(s): 811 - 814
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (98 KB)  

    In their original paper, C.-C. Ku and K.Y. Lee (ibid., vol.6, p.144-56, 1995) designed a diagonal recurrent neural network architecture for control systems. Liang asserts that a condition assumed in the proof of its convergence does not necessarily apply, and presents alternative theorems and proofs. Lee replies that Liang has misunderstood the original paper, and also that he made mistakes in his new proofs, but acknowledges that the original paper did not go into full detail about how a limitation would be implemented. Lee also presents a revision for the case of time-varying weights. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author's reply And Revision For Time-varying Weights

    Publication Year: 1997 , Page(s): 813 - 814
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (67 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Pattern Recognition And Neural Networks [Book Reviews]

    Publication Year: 1997 , Page(s): 815 - 816
    Save to Project icon | Request Permissions | PDF file iconPDF (24 KB)  
    Freely Available from IEEE
  • Neural Network Design [Books in Brief]

    Publication Year: 1997 , Page(s): 817
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • Computational Intelligence Pc Tools [Books in Brief]

    Publication Year: 1997 , Page(s): 817
    Save to Project icon | Request Permissions | PDF file iconPDF (7 KB)  
    Freely Available from IEEE
  • SOIM: a self-organizing invertible map with applications in active vision

    Publication Year: 1997 , Page(s): 758 - 773
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (376 KB)  

    We propose a novel neural network, called the self-organized invertible map (SOIM), that is capable of learning many-to-one functionals mappings in a self-organized and online fashion. The design and performance of the SOIM are highlighted by learning a many-to-one functional mapping that exists in active vision for spatial representation of three-dimensional point targets. The learned spatial representation is invariant to changing camera configurations. The SOIM also possesses an invertible property that can be exploited for active vision. An efficient and experimentally feasible method was devised for learning this representation on a real active vision system. The proof of convergence during learning as well as conditions for invariance of the learned spatial representation are derived and then experimentally verified using the active vision system. We also demonstrate various active vision applications that benefit from the properties of the mapping learned by SOIM View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural networks for convex hull computation

    Publication Year: 1997 , Page(s): 601 - 611
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (496 KB)  

    Computing convex hull is one of the central problems in various applications of computational geometry. In this paper, a convex hull computing neural network (CHCNN) is developed to solve the related problems in the N-dimensional spaces. The algorithm is based on a two-layered neural network, topologically similar to ART, with a newly developed adaptive training strategy called excited learning. The CHCNN provides a parallel online and real-time processing of data which, after training, yields two closely related approximations, one from within and one from outside, of the desired convex hull. It is shown that accuracy of the approximate convex hulls obtained is around O[K-1(N-1/)], where K is the number of neurons in the output layer of the CHCNN. When K is taken to be sufficiently large, the CHCNN can generate any accurate approximate convex hull. We also show that an upper bound exists such that the CHCNN will yield the precise convex hull when K is larger than or equal to this bound. A series of simulations and applications is provided to demonstrate the feasibility, effectiveness, and high efficiency of the proposed algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Supervised learning of perceptron and output feedback dynamic networks: a feedback analysis via the small gain theorem

    Publication Year: 1997 , Page(s): 612 - 622
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (424 KB)  

    This paper provides a time-domain feedback analysis of the perceptron learning algorithm and of training schemes for dynamic networks with output feedback. It studies the robustness performance of the algorithms in the presence of uncertainties that might be due to noisy perturbations in the reference signals or due to modeling mismatch. In particular, bounds are established on the step-size parameters in order to guarantee that the resulting algorithms will behave as robust filters. The paper also establishes that an intrinsic feedback structure can be associated with the training schemes. The feedback configuration is motivated via energy arguments and is shown to consist of two major blocks: a time-variant lossless (i.e., energy preserving) feedforward path and a time-variant feedback path. The stability of the feedback structure is then analyzed via the small gain theorem, and choices for the step-size parameter in order to guarantee faster convergence are deduced by using the mean-value theorem. Simulation results are included to demonstrate the findings View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Rotation-invariant neural pattern recognition system estimating a rotation angle

    Publication Year: 1997 , Page(s): 568 - 581
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (348 KB)  

    A rotation-invariant neural pattern recognition system, which can recognize a rotated pattern and estimate its rotation angle, is considered. It is well-known that humans sometimes recognize a rotated form by means of mental rotation. The occurrence of mental rotation can be explained in terms of the theory of information types. Therefore, we first examine the applicability of the theory to a rotation-invariant neural pattern recognition system. Next, we present a rotation-invariant neural network which can estimate a rotation angle. The neural network consists of a preprocessing network to detect the edge features of input patterns and a trainable multilayered network. Furthermore, a rotation-invariant neural pattern recognition system which includes the rotation-invariant neural network is proposed. This system is constructed on the basis of the above-mentioned theory. Finally, it is shown that, by means of computer simulations of a binary pattern and a coin recognition problem, the system is able to recognize rotated patterns and estimate their rotation angle View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An improved recurrent neural network for M-PAM symbol detection

    Publication Year: 1997 , Page(s): 779 - 783
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (168 KB)  

    In this paper, a fully connected recurrent neural network (RNN) is presented for the recovery of M-ary pulse amplitude modulated (M-PAM) signals in the presence of intersymbol interference and additive white Gaussian noise. The network makes use of two different activation functions. One is the traditional two-level sigmoid function, which is used at its hidden nodes, and the other is the M-level sigmoid function (MSF), which is used at the output node. The shape of the M-level activation function is controlled by two parameters: the slope and shifting parameters. The effect of these parameters on the learning performance is investigated through extensive simulations. In addition, the network is compared with a linear transversal equalizer, a decision feedback equalizer and a recently proposed RNN equalizer which has used a scaled sigmoid function (SSF) at its output node. Comparisons are made in terms of their learning properties and symbol error rates. It is demonstrated that the proposed RNN equalizer performs better, provided that the MSF parameters are properly selected View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Extended least squares based algorithm for training feedforward networks

    Publication Year: 1997 , Page(s): 806 - 810
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (148 KB)  

    An extended least squares-based algorithm for feedforward networks is proposed. The weights connecting the last hidden and output layers are first evaluated by least squares algorithm. The weights between input and hidden layers are then evaluated using the modified gradient descent algorithms. This arrangement eliminates the stalling problem experienced by the pure least squares type algorithms; however, still maintains the characteristic of fast convergence. In the investigated problems, the total number of FLOPS required for the networks to converge using the proposed training algorithm are only 0.221%-16.0% of that using the Levenberg-Marquardt algorithm. The number of floating point operations per iteration of the proposed algorithm are only 1.517-3.521 times of that of the standard backpropagation algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Acquiring rule sets as a product of learning in a logical neural architecture

    Publication Year: 1997 , Page(s): 461 - 474
    Cited by:  Papers (24)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (312 KB)  

    Envisioning neural networks as systems that learn rules calls forth the verification issues already being studied in knowledge-based systems engineering, and complicates these with neural-network concepts such as nonlinear dynamics and distributed memories. We show that the issues can be clarified and the learned rules visualized symbolically by formalizing the semantics of rule-learning in the mathematical language of two-valued predicate logic. We further show that this can, at least in some cases, be done with a fairly simple logical model. We illustrate this with a combination of two example neural-network architectures, LAPART, designed to learn rules as logical inferences from binary data patterns, and the stack interval network, which converts real-valued data into binary patterns that preserve the semantics of the ordering of real values. We discuss the significance of the formal model in facilitating the analysis of the underlying logic of rule-learning and numerical data representation. We provide examples to illustrate the formal model, with the combined stack interval/LAPART networks extracting rules from numerical data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Hierarchical graph visualization using neural networks

    Publication Year: 1997 , Page(s): 794 - 799
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (200 KB)  

    An algorithm based on a Hopfield network for solving the hierarchical graph visualization problem is presented. It simultaneously minimizes the number of crossings and total path length to produce two-dimensional drawings easily interpreted by human observers. Traditional heuristics often follow a more local optimization approach where “readability” criteria are sequentially applied, such as applying the barycentric heuristic followed by the priority layout heuristic. As a result of the more global approach, the neural network achieved comparable crossing minimization to the barycentric heuristic while simultaneously reducing total path length up to 50% over the priority layout heuristic for the benchmarks tested View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonlinear control structures based on embedded neural system models

    Publication Year: 1997 , Page(s): 553 - 567
    Cited by:  Papers (51)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (448 KB)  

    This paper investigates in detail the possible application of neural networks to the modeling and adaptive control of nonlinear systems. Nonlinear neural-network-based plant modeling is first discussed, based on the approximation capabilities of the multilayer perceptron. A structure is then proposed to utilize feedforward networks within a direct model reference adaptive control strategy. The difficulties involved in training this network, embedded within the closed-loop are discussed and a novel neural-network-based sensitivity modeling approach proposed to allow for the backpropagation of errors through the plant to the neural controller. Finally, a novel nonlinear internal model control (IMC) strategy is suggested, that utilizes a nonlinear neural model of the plant to generate parameter estimates over the nonlinear operating region for an adaptive linear internal model, without the problems associated with recursive parameter identification algorithms. Unlike other neural IMC approaches the linear control law can then be readily designed. A continuous stirred tank reactor was chosen as a realistic nonlinear case study for the techniques discussed in the paper View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new approach to Kanerva's sparse distributed memory

    Publication Year: 1997 , Page(s): 791 - 794
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (100 KB)  

    The sparse distributed memory (SDM) was originally developed to tackle the problem of storing large binary data patterns. The model succeeded well in storing random input data. However, its efficiency, particularly in handling nonrandom data, was poor. In its original form it is a static and inflexible system. Most of the recent work on the SDM has concentrated on improving the efficiency of a modified form of the SDM which treats the memory as a single-layer neural network. This paper introduces an alternative SDM, the SDM signal model which retains the essential characteristics of the original SDM, while providing the memory with a greater scope for plasticity and self-evolution. By removing many of the problematic features of the original SDM the new model is not as dependent upon a priori input values. This gives it an increased robustness to learn either random or correlated input patterns. The improvements in this new SDM signal model should be also of benefit to modified SDM neural network models View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On solving systems of linear inequalities with artificial neural networks

    Publication Year: 1997 , Page(s): 590 - 600
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    The implementation of the relaxation-projection algorithm by artificial neural networks to solve sets of linear inequalities is examined. The different versions of this algorithm are described, and theoretical convergence results are given. The best known analog optimization solvers are shown to use the simultaneous projection version of it. Neural networks that implement each version are described. The results of tests, made with simulated realizations of these networks, are reported. These tests consisted in having all networks solve some sample problems. The results obtained help determine good values for the step size parameters, and point out the relative merits of the different networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fast parallel off-line training of multilayer perceptrons

    Publication Year: 1997 , Page(s): 646 - 653
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (184 KB)  

    Various approaches to the parallel implementation of second-order gradient-based multilayer perceptron training algorithms are described. Two main classes of algorithm are defined involving Hessian and conjugate gradient-based methods. The limited- and full-memory Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithms are selected as representative examples and used to show that the step size and gradient calculations are critical components. For larger problems the matrix calculations in the full-memory algorithm are also significant. Various strategies are considered for parallelization, the best of which is implemented on parallel virtual machine (PVM) and transputer-based architectures. Results from a range of problems are used to demonstrate the performance achievable with each architecture. The transputer implementation is found to give excellent speed-ups but the problem size is limited by memory constraints. The speed-ups achievable with the PVM implementation are much poorer because of inefficient communication, but memory is not a difficulty View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Real-time classification of rotating shaft loading conditions using artificial neural networks

    Publication Year: 1997 , Page(s): 748 - 757
    Cited by:  Papers (18)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB)  

    Vibration analysis can give an indication of the condition of a rotating shaft highlighting potential faults such as unbalance and rubbing. Faults may however only occur intermittently and consequently to detect these requires continuous monitoring with real time analysis. This paper describes the use of artificial neural networks (ANNs) for classification of condition and compares these with other discriminant analysis methods. Moments calculated from time series are used as input features as they can be quickly computed from the measured data. Orthogonal vibrations are considered as a two-dimensional vector, the magnitude of which can be expressed as time series. Some simple signal processing operations are applied to the data to enhance the differences between signals and comparison is made with frequency domain analysis View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Supervised neural networks for the classification of structures

    Publication Year: 1997 , Page(s): 714 - 735
    Cited by:  Papers (83)  |  Patents (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (660 KB)  

    Standard neural networks and statistical methods are usually believed to be inadequate when dealing with complex structures because of their feature-based approach. In fact, feature-based approaches usually fail to give satisfactory solutions because of the sensitivity of the approach to the a priori selection of the features, and the incapacity to represent any specific information on the relationships among the components of the structures. However, we show that neural networks can, in fact, represent and classify structured patterns. The key idea underpinning our approach is the use of the so called “generalized recursive neuron”, which is essentially a generalization to structures of a recurrent neuron. By using generalized recursive neurons, all the supervised networks developed for the classification of sequences, such as backpropagation through time networks, real-time recurrent networks, simple recurrent networks, recurrent cascade correlation networks, and neural trees can, on the whole, be generalized to structures. The results obtained by some of the above networks (with generalized recursive neurons) on the classification of logic terms are presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive control using neural networks and approximate models

    Publication Year: 1997 , Page(s): 475 - 485
    Cited by:  Papers (147)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (416 KB)  

    The NARMA model is an exact representation of the input-output behavior of finite-dimensional nonlinear discrete-time dynamical systems in a neighborhood of the equilibrium state. However, it is not convenient for purposes of adaptive control using neural networks due to its nonlinear dependence on the control input. Hence, quite often, approximate methods are used for realizing the neural controllers to overcome computational complexity. In this paper, we introduce two classes of models which are approximations to the NARMA model, and which are linear in the control input. The latter fact substantially simplifies both the theoretical analysis as well as the practical implementation of the controller. Extensive simulation studies have shown that the neural controllers designed using the proposed approximate models perform very well, and in many cases even better than an approximate controller designed using the exact NARMA model. In view of their mathematical tractability as well as their success in simulation studies, a case is made in this paper that such approximate input-output models warrant a detailed study in their own right View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constructive algorithms for structure learning in feedforward neural networks for regression problems

    Publication Year: 1997 , Page(s): 630 - 645
    Cited by:  Papers (122)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole problem as a state-space search, we first describe the general issues in constructive algorithms, with special emphasis on the search strategy. A taxonomy, based on the differences in the state transition mapping, the training algorithm, and the network architecture, is then presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Primal and dual assignment networks

    Publication Year: 1997 , Page(s): 784 - 790
    Cited by:  Papers (20)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (248 KB)  

    This paper presents two recurrent neural networks for solving the assignment problem. Simplifying the architecture of a recurrent neural network based on the primal assignment problem, the first recurrent neural network, called the primal assignment network, has less complex connectivity than its predecessor. The second recurrent neural network, called the dual assignment network, based on the dual assignment problem, is even simpler in architecture than the primal assignment network. The primal and dual assignment networks are guaranteed to make optimal assignment. The applications of the primal and dual assignment networks for sorting and shortest-path routing are discussed. The performance and operating characteristics of the dual assignment network are demonstrated by means of illustrative examples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On convergence properties of pocket algorithm

    Publication Year: 1997 , Page(s): 623 - 629
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    The problem of finding optimal weights for a single threshold neuron starting from a general training set is considered. Among the variety of possible learning techniques, the pocket algorithm has a proper convergence theorem which asserts its optimality. However, the original proof ensures the asymptotic achievement of an optimal weight vector only if the inputs in the training set are integer or rational. This limitation is overcome in this paper by introducing a different approach that leads to the general result. Furthermore, a modified version of the learning method considered, called pocket algorithm with ratchet, is shown to obtain an optimal configuration within a finite number of iterations independently of the given training set View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A methodology for constructing fuzzy algorithms for learning vector quantization

    Publication Year: 1997 , Page(s): 505 - 518
    Cited by:  Papers (31)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (612 KB)  

    This paper presents a general methodology for the development of fuzzy algorithms for learning vector quantization (FALVQ). The design of specific FALVQ algorithms according to existing approaches reduces to the selection of the membership function assigned to the weight vectors of an LVQ competitive neural network, which represent the prototypes. The development of a broad variety of FALVQ algorithms can be accomplished by selecting the form of the interference function that determines the effect of the nonwinning prototypes on the attraction between the winning prototype and the input of the network. The proposed methodology provides the basis for extending the existing FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms. This paper also introduces two quantitative measures which establish a relationship between the formulation that led to FALVQ algorithms and the competition between the prototypes during the learning process. The proposed algorithms and competition measures are tested and evaluated using the IRIS data set. The significance of the proposed competition measure is illustrated using FALVQ algorithms to perform segmentation of magnetic resonance images of the brain View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An iterative pruning algorithm for feedforward neural networks

    Publication Year: 1997 , Page(s): 519 - 531
    Cited by:  Papers (76)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (340 KB)  

    The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope