Scheduled System Maintenance on May 29th, 2015:
IEEE Xplore will be upgraded between 11:00 AM and 10:00 PM EDT. During this time there may be intermittent impact on performance. We apologize for any inconvenience.
By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 1997

Filter Results

Displaying Results 1 - 25 of 33
  • Comments on "A self-organizing network for hyperellipsoidal clustering (HEC)" [with reply]

    Publication Year: 1997 , Page(s): 1561 - 1563
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (68 KB)  

    In the above paper by Mao-Jain (ibid., vol.7 (1996)), the Mahalanobis distance is used instead of Euclidean distance as the distance measure in order to acquire the hyperellipsoidal clustering. We prove that the clustering cost function is a constant under this condition, so hyperellipsoidal clustering cannot be realized. We also explains why the clustering algorithm developed in the above paper can get some good hyperellipsoidal clustering results. In reply, Mao-Jain state that the Wang-Xia failed to point out that their HEC clustering algorithm used a regularized Mahalanobis distance instead of the standard Mahalanobis distance. It is the regularized Mahalanobis distance which plays an important role in realizing hyperellipsoidal clusters. In conclusion, the comments made by Wang-Xia together with this response provide some new insights into the behavior of their HEC clustering algorithm. It further confirms that the HEC algorithm is a useful tool for understanding the structure of multidimensional data. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author's reply

    Publication Year: 1997 , Page(s): 1563
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (27 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Corrections To "Adaptive Critic Designs"

    Publication Year: 1997 , Page(s): 1563
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (27 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Power prediction in mobile communication systems using an optimal neural-network structure

    Publication Year: 1997 , Page(s): 1446 - 1455
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (234 KB)  

    Presents a novel neural-network-based predictor for received power level prediction in direct sequence code division multiple access (DS/CDMA) systems. The predictor consists of an adaptive linear element (Adaline) followed by a multilayer perceptron (MLP). An important but difficult problem in designing such a cascade predictor is to determine the complexity of the networks. We solve this problem by using the predictive minimum description length (PMDL) principle to select the optimal numbers of input and hidden nodes. This approach results in a predictor with both good noise attenuation and excellent generalization capability. The optimized neural networks are used for predictive filtering of very noisy Rayleigh fading signals with 1.8 GHz carrier frequency. Our results show that the optimal neural predictor can provide smoothed in-phase and quadrature signals with signal-to-noise ratio (SNR) gains of about 12 and 7 dB at the urban mobile speeds of 5 and 50 km/h, respectively. The corresponding power signal SNR gains are about 11 and 5 dB. Therefore, the neural predictor is well suitable for power control applications where ldquodelaylessrdquo noise attenuation and efficient reduction of fast fading are required View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • k-winners-take-all neural net with Θ(1) time complexity

    Publication Year: 1997 , Page(s): 1557 - 1561
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (164 KB)  

    In this article we present a k-winners-take-all (k-WTA) neural net that is established based on the concept of the constant time sorting machine by Hsu and Wang. It fits some specific applications, such as real-time processing, since its Θ(1) time complexity is independent to the problem size. The proposed k-WTA neural net produces the solution in constant time while the Hopfield network requires a relatively long transient to converge to the solution from some initial states View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Mine detection using scattering parameters

    Publication Year: 1997 , Page(s): 1456 - 1467
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (228 KB)  

    The detection and disposal of antipersonnel land mines is one of the most difficult and intractable problems faced in ground conflict. This paper presents detection methods which use a separated-aperture microwave sensor and an artificial neural network pattern classifier. Several data-specific preprocessing methods are developed to enhance neural network learning. In addition, a generalized Karhunen-Loeve transform and the eigenspace separation transform are used to perform data reduction and reduce network complexity. Highly favorable results have been obtained using the above methods in conjunction with a feedforward neural network View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Structure of the high-order Boltzmann machine from independence maps

    Publication Year: 1997 , Page(s): 1351 - 1358
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (316 KB)  

    In this paper we consider the determination of the structure of the high-order Boltzmann machine (HOBM), a stochastic recurrent network for approximating probability distributions. We obtain the structure of the HOBM, the hypergraph of connections, from conditional independences of the probability distribution to model. We assume that an expert provides these conditional independences and from them we build independence maps, Markov and Bayesian networks, which represent conditional independences through undirected graphs and directed acyclic graphs respectively. From these independence maps we construct the HOBM hypergraph. The central aim of this paper is to obtain a minimal hypergraph. Given that different orderings of the variables provide in general different Bayesian networks, we define their intersection hypergraph. We prove that the intersection hypergraph of all the Bayesian networks (N!) of the distribution is contained by the hypergraph of the Markov network, it is more simple, and we give a procedure to determine a subset of the Bayesian networks that verifies this property. We also prove that the Markov network graph establishes a minimum connectivity for the hypergraphs from Bayesian networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonlinear backpropagation: doing backpropagation without derivatives of the activation function

    Publication Year: 1997 , Page(s): 1321 - 1327
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (172 KB)  

    The conventional linear backpropagation algorithm is replaced by a nonlinear version, which avoids the necessity for calculating the derivative of the activation function. This may be exploited in hardware realizations of neural processors. In this paper we derive the nonlinear backpropagation algorithms in the framework of recurrent backpropagation and present some numerical simulations of feedforward networks on the NetTalk problem. A discussion of implementation in analog very large scale integration (VLSI) electronics concludes the paper View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new synthesis approach for feedback neural networks based on the perceptron training algorithm

    Publication Year: 1997 , Page(s): 1468 - 1482
    Cited by:  Papers (25)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (584 KB)  

    In this paper, a new synthesis approach is developed for associative memories based on the perceptron training algorithm. The design (synthesis) problem of feedback neural networks for associative memories is formulated as a set of linear inequalities such that the use of perceptron training is evident. The perceptron training in the synthesis algorithms is guaranteed to converge for the design of neural networks without any constraints on the connection matrix. For neural networks with constraints on the diagonal elements of the connection matrix, results concerning the properties of such networks and concerning the existence of such a network design are established. For neural networks with sparsity and/or symmetry constraints on the connection matrix, design algorithms are presented. Applications of the present synthesis approach to the design of associative memories realized by means of other feedback neural network models are studied. To demonstrate the applicability of the present results and to compare the present synthesis approach with existing design methods, specific examples are considered View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Self-organizing algorithms for generalized eigen-decomposition

    Publication Year: 1997 , Page(s): 1518 - 1530
    Cited by:  Papers (23)  |  Patents (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (612 KB)  

    We discuss a new approach to self-organization that leads to novel adaptive algorithms for generalized eigen-decomposition and its variance for a single-layer linear feedforward neural network. First, we derive two novel iterative algorithms for linear discriminant analysis (LDA) and generalized eigen-decomposition by utilizing a constrained least-mean-squared classification error cost function, and the framework of a two-layer linear heteroassociative network performing a one-of-m classification. By using the concept of deflation, we are able to find sequential versions of these algorithms which extract the LDA components and generalized eigenvectors in a decreasing order of significance. Next, two new adaptive algorithms are described to compute the principal generalized eigenvectors of two matrices (as well as LDA) from two sequences of random matrices. We give a rigorous convergence analysis of our adaptive algorithms by using stochastic approximation theory, and prove that our algorithms converge with probability one View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Development and application of an integrated neural system for an HDCL

    Publication Year: 1997 , Page(s): 1328 - 1337
    Cited by:  Papers (3)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (228 KB)  

    This study presents the development and industrial application of an integrated neural system in coating weight control for a modern hot dip coating line (HDCL) in a steel mill. The neural system consists of two multilayered feedforward neural networks and a neural adaptive controller. They perform coating weight real-time prediction, feedforward control (FFC), and adaptive feedback control (FBC), respectively. The production line analysis, neural system architecture, learning, associative memories, generalization and real-time applications are addressed in this paper. This integrated neural system has been successfully implemented and applied to an HDCL at Burns Harbor Division, Bethlehem Steel Co., Chesterton, IN. The industrial application results have shown significant improvements in reduction of coating weight transitional footage, variation of the error between the target and actual coating weight, and the coating material used. Some practical aspects for applying a neural system to industrial control are discussed as concluding remarks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fast training of multilayer perceptrons

    Publication Year: 1997 , Page(s): 1314 - 1320
    Cited by:  Papers (30)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (140 KB)  

    Training a multilayer perceptron by an error backpropagation algorithm is slow and uncertain. This paper describes a new approach which is much faster and certain than error backpropagation. The proposed approach is based on combined iterative and direct solution methods. In this approach, we use an inverse transformation for linearization of nonlinear output activation functions, direct solution matrix methods for training the weights of the output layer; and gradient descent, the delta rule, and other proposed techniques for training the weights of the hidden layers. The approach has been implemented and tested on many problems. Experimental results, including training times and recognition accuracy, are given. Generally, the approach achieves accuracy as good as or better than perceptrons trained using error backpropagation, and the training process is much faster than the error backpropagation algorithm and also avoids local minima and paralysis View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stable online evolutionary learning of NN-MLP

    Publication Year: 1997 , Page(s): 1371 - 1378
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (196 KB)  

    To design the nearest-neighbor-based multilayer perceptron (NN-MLP) efficiently, the author has proposed a nongenetic-based evolutionary algorithm called the R4-rule. For off-line learning, the R4-rule can produce the smallest or nearly smallest networks with high generalization ability by iteratively performing four basic operations: recognition, remembrance, reduction, add review. This algorithm, however, cannot be applied directly to online learning because its inherent instability, which is caused by over-reduction and over-review. To stabilize the R4-rule, this paper proposes some improvements for reduction and review. The improved reduction is more robust for online learning because the fitness of each hidden neuron is defined by its overall behavior in many learning cycles. The new review is more efficient because hidden neurons are adjusted in a more careful way. The performance of the improved R 4-rule for online learning is shown by experimental results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques

    Publication Year: 1997 , Page(s): 1492 - 1506
    Cited by:  Papers (99)  |  Patents (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (528 KB)  

    This paper proposes a framework for constructing and training radial basis function (RBF) neural networks. The proposed growing radial basis function (GRBF) network begins with a small number of prototypes, which determine the locations of radial basis functions. In the process of training, the GRBF network gross by splitting one of the prototypes at each growing cycle. Two splitting criteria are proposed to determine which prototype to split in each growing cycle. The proposed hybrid learning scheme provides a framework for incorporating existing algorithms in the training of GRBF networks. These include unsupervised algorithms for clustering and learning vector quantization, as well as learning algorithms for training single-layer linear neural networks. A supervised learning scheme based on the minimization of the localized class-conditional variance is also proposed and tested. GRBF neural networks are evaluated and tested on a variety of data sets with very satisfactory results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A partial order for the M-of-N rule-extraction algorithm

    Publication Year: 1997 , Page(s): 1542 - 1544
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (180 KB)  

    We present a method to unify the rules obtained by the M-of-N rule-extraction technique. The rules extracted from a perceptron by the M-of-N algorithm are in correspondence with sets of minimal Boolean vectors with respect to the classical partial order defined on vectors. Our method relies on a simple characterization of another partial order defined on Boolean vectors. We show that there exists also a correspondence between sets of minimal Boolean vectors with respect to this order and M-of-N rules equivalent to a perceptron. The gain is that fewer rules are generated with the second order. Independently, we prove that deciding whether a perceptron is symmetric with respect to two variables is NP-complete View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the distribution of performance from multiple neural-network trials

    Publication Year: 1997 , Page(s): 1507 - 1517
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    The performance of neural network simulations is often reported in terms of the mean and standard deviation of a number of simulations performed with different starting conditions. However, in many cases, the distribution of the individual results does not approximate a Gaussian distribution, may not be symmetric, and may be multimodal. We present the distribution of results for practical problems and show that assuming Gaussian distributions can significantly affect the interpretation of results, especially those of comparison studies. For a controlled task which we consider, we find that the distribution of performance is skewed toward better performance for smoother target functions and skewed toward worse performance for more complex target functions. We propose new guidelines for reporting performance which provide more information about the actual distribution View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Reduction of breast biopsies with a modified self-organizing map

    Publication Year: 1997 , Page(s): 1386 - 1396
    Cited by:  Papers (18)  |  Patents (14)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (428 KB)  

    A modified self-organizing map with nonlinear weight adjustments has been applied to reduce the number of breast biopsies necessary for breast cancer diagnosis. Tissue features representing texture information from digital sonographic breast images were extracted from sonograms of benign and malignant breast tumors. The resulting hyperspace of data points was then used in a modified self-organizing map that objectively segments population distributions of lesions and accurately establishes benign and malignant regions. These methods were applied to a group of 102 problematic breast cases with sonographic images, including 34 with malignant lesions. All lesions were substantiated by excisional biopsy. The system can isolate clusters of purely benign lesions from other clusters containing both benign and malignant lesions. The hybrid neural network defined a region in which about 60% of the benign lesions were located exclusive of any malignant lesions. The experimental results also suggest that the modified self-organizing map provides more accurate population distribution maps than conventional Kohonen maps View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural networks in financial engineering: a study in methodology

    Publication Year: 1997 , Page(s): 1222 - 1267
    Cited by:  Papers (42)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1912 KB)  

    Neural networks have shown considerable successes in modeling financial data series. However, a major weakness of neural modeling is the lack of established procedures for performing tests for misspecified models, and tests of statistical significance for the various parameters that have been estimated. This is a serious disadvantage in applications where there is a strong culture for testing not only the predictive power of a model or the sensitivity of the dependent variable to changes in the inputs but also the statistical significance of the finding at a specified level of confidence. Rarely is this more important than in the case of financial engineering, where the data generating processes are dominantly stochastic and only partially deterministic. Partly a tutorial, partly a review, this paper describes a collection of typical applications in options pricing, cointegration, the term structure of interest rates and models of investor behavior which highlight these weaknesses and propose and evaluate a number of solutions. We describe a number of alternative ways to deal with the problem of variable selection, show how to use model misspecification tests, we deploy a novel way based on cointegration to deal with the problem of nonstationarity, and generally describe approaches to predictive neural modeling which are more in tune with the requirements for modeling financial data series View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neural-network learning theory and a polynomial time RBF algorithm

    Publication Year: 1997 , Page(s): 1301 - 1313
    Cited by:  Papers (28)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (336 KB)  

    This paper presents a new learning theory (a set of principles for brain-like learning) and a corresponding algorithm for the neural-network field. The learning theory defines computational characteristics that are much more brain-like than that of classical connectionist learning. Robust and reliable learning algorithms would result if these learning principles are followed rigorously when developing neural-network algorithms. This paper also presents a new algorithm for generating radial basis function (RBF) nets for function approximation. The design of the algorithm is based on the proposed set of learning principles. The net generated by this algorithm is not a typical RBF net, but a combination of “truncated” RBF and other types of hidden units. The algorithm uses random clustering and linear programming (LP) to design and train this “mixed” RBF net. Polynomial time complexity of the algorithm is proven and computational results are provided for the well known Mackey-Glass chaotic time series problem, the logistic map prediction problem, various neuro-control problems, and several time series forecasting problems. The algorithm can also be implemented as an online adaptive algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Knowledge-based fuzzy MLP for classification and rule generation

    Publication Year: 1997 , Page(s): 1338 - 1350
    Cited by:  Papers (20)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (360 KB)  

    A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning convergence of CMAC technique

    Publication Year: 1997 , Page(s): 1281 - 1292
    Cited by:  Papers (50)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (568 KB)  

    CMAC is one useful learning technique that was developed two decades ago but yet lacks adequate theoretical foundation. Most past studies focused on development of algorithms, improvement of the CMAC structure, and applications. Given a learning problem, very little about the CMAC learning behavior such as the convergence characteristics, effects of hash mapping, effects of memory size, the error bound, etc. can be analyzed or predicted. In this paper, we describe the CMAC technique with mathematical formulation and use the formulation to study the CMAC convergence properties. Both information retrieval and learning rules are described by algebraic equations in matrix form. Convergence characteristics and learning behaviors for the CMAC with and without hash mapping are investigated with the use of these equations and eigenvalues of some derived matrices. The formulation and results provide a foundation for further investigation of this technique View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The Nature Of Statistical Learning Theory~

    Publication Year: 1997 , Page(s): 1564
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (16 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A gradual neural-network approach for frequency assignment in satellite communication systems

    Publication Year: 1997 , Page(s): 1359 - 1370
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB)  

    A novel neural-network approach called gradual neural network (GNN) is presented for a class of combinatorial optimization problems of requiring the constraint satisfaction and the goal function optimization simultaneously. The frequency assignment problem in the satellite communication system is efficiently solved by GNN as the typical problem of this class. The goal of this NP-complete problem is to minimize the cochannel interference between satellite communication systems by rearranging the frequency assignment so that they can accommodate the increasing demands. The GNN consists of N×M binary neurons for the N-carrier-M-segment system with the gradual expansion scheme of activated neurons. The binary neural network achieves the constrain satisfaction with the help of heuristic methods, whereas the gradual expansion scheme seeks the cost optimization. The capability of GNN is demonstrated through solving 15 instances in practical size systems, where GNN can find far better solutions than the existing algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recurrent neural nets as dynamical Boolean systems with application to associative memory

    Publication Year: 1997 , Page(s): 1268 - 1280
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (496 KB)  

    Discrete-time/discrete-state recurrent neural networks are analyzed from a dynamical Boolean systems point of view in order to devise new analytic and design methods for the class of both single and multilayer recurrent artificial neural networks. With the proposed dynamical Boolean systems analysis, we are able to formulate necessary and sufficient conditions for network stability which are more general than the well-known but restrictive conditions for the class of single layer networks: (1) symmetric weight matrix with (2) positive diagonal and (3) asynchronous update. In terms of design, we use a dynamical Boolean systems analysis to construct a high performance associative memory. With this Boolean memory, we can guarantee that all fundamental memories are stored, and also guarantee the size of the basin of attraction for each fundamental memory View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A knowledge-base generating hierarchical fuzzy-neural controller

    Publication Year: 1997 , Page(s): 1531 - 1541
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (260 KB)  

    We present an innovative fuzzy-neural architecture that is able to automatically generate a knowledge base, in an extractable form, for use in hierarchical knowledge-based controllers. The knowledge base is in the form of a linguistic rule base appropriate for a fuzzy inference system. First, we modify Berenji and Khedkar's (1992) GARIC architecture to enable it to automatically generate a knowledge base; a pseudosupervised learning scheme using reinforcement learning and error backpropagation is employed. Next, we further extend this architecture to a hierarchical controller that is able to generate its own knowledge base. Example applications are provided to underscore its viability View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope