By Topic

Neural Networks, IEEE Transactions on

Issue 1 • Date Jan. 1998

Filter Results

Displaying Results 1 - 25 of 27
  • Elements Of Artificial Neural Networks [Book Reviews]

    Page(s): 234 - 235
    Save to Project icon | Request Permissions | PDF file iconPDF (17 KB)  
    Freely Available from IEEE
  • Neural Network Analysis, Architectures And Applications [Books in Brief]

    Page(s): 236
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • A bootstrap evaluation of the effect of data splitting on financial time series

    Page(s): 213 - 220
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (172 KB)  

    Exposes problems of the commonly used technique of splitting the available data into training, validation, and test sets that are held fixed, warns about drawing too strong conclusions from such static splits, and shows potential pitfalls of ignoring variability across splits. Using a bootstrap or resampling method, we compare the uncertainty in the solution stemming from the data splitting with neural-network specific uncertainties (parameter initialization, choice of number of hidden units, etc.). We present two results on data from the New York Stock Exchange. First, the variation due to different resamplings is significantly larger than the variation due to different network conditions. This result implies that it is important to not over-interpret a model (or an ensemble of models) estimated on one specific split of the data. Second, on each split, the neural-network solution with early stopping is very close to a linear model; no significant nonlinearities are extracted View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Glove-TalkII-a neural-network interface which maps gestures to parallel formant speech synthesizer controls

    Page(s): 205 - 212
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (152 KB)  

    Glove-TalkII is a system which translates hand gestures to speech through an adaptive interface. Hand gestures are mapped continuously to ten control parameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-TalkII uses several input devices (including a Cyberglove, a ContactGlove, a three-space tracker, and a foot pedal), a parallel formant speech synthesizer, and three neural networks. The gesture-to-speech task is divided into vowel and consonant production by using a gating network to weight the outputs of a vowel and a consonant neural network. The gating network and the consonant network are trained with examples from the user. The vowel network implements a fixed user-defined relationship between hand position and vowel sound and does not require any training examples from the user. Volume, fundamental frequency, and stop consonants are produced with a fixed mapping from the input devices. One subject has trained to speak intelligibly with Glove-TalkII. He speaks slowly but with far more natural sounding pitch variations than a text-to-speech synthesizer View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability analysis for neural dynamics with time-varying delays

    Page(s): 221 - 223
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    By using the usual additive neural-network model, a delay-independent stability criterion for neural dynamics with perturbations of time-varying delays is derived. We extend previously known results obtained by Gopalsamy and He (1994) to the time varying delay case, and present decay estimates of solutions of neural networks. The asymptotic stability is global in the state space of neuronal activations. From the techniques used in this paper, it is shown that our criterion ensures stability of neural dynamics even when the delay functions vary violently with time. Our approach provides an effective method for the stability analysis of neural dynamics with delays View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • MART: a multichannel ART-based neural network

    Page(s): 139 - 150
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (312 KB)  

    This paper describes MART, an ART-based neural network for adaptive classification of multichannel signal patterns without prior supervised learning. Like other ART-based classifiers, MART is especially suitable for situations in which not even the number of pattern categories to be distinguished is known a priori; its novelty lies in its truly multichannel orientation, especially its ability to quantify and take into account during pattern classification the different changing reliability of the individual signal channels. The extent to which this ability can reduce the creation of spurious or duplicate categories (a major problem for ART-based classifiers of noisy signals) is illustrated by evaluation of its performance in classifying QRS complexes in two-channel ECG traces which were taken from the MIT-BIH database and contaminated with noise View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions

    Page(s): 224 - 229
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (220 KB)  

    It is well known that standard single-hidden layer feedforward networks (SLFNs) with at most N hidden neurons (including biases) can learn N distinct samples (xi,ti) with zero error, and the weights connecting the input neurons and the hidden neurons can be chosen “almost” arbitrarily. However, these results have been obtained for the case when the activation function for the hidden neurons is the signum function. This paper rigorously proves that standard single-hidden layer feedforward networks (SLFNs) with at most N hidden neurons and with any bounded nonlinear activation function which has a limit at one infinity can learn N distinct samples (xi,ti) with zero error. The previous method of arbitrarily choosing weights is not feasible for any SLFN. The proof of our result is constructive and thus gives a method to directly find the weights of the standard SLFNs with any such bounded nonlinear activation function as opposed to iterative training algorithms in the literature View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A CMOS binary pattern classifier based on Parzen's method

    Page(s): 2 - 10
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (352 KB)  

    Biological circuitry in the brain that has been associated with the Parzen method of classification inspired an analog CMOS binary pattern classifier. The circuitry resides on three separate chips. The first chip computes the closeness of a test vector to each training vector stored on the chip where “vector closeness” is defined as the number of bits two vectors have in common above some thresholds. The second chip computes the closeness of the test vector to each possible category where “category closeness” is defined as the sum of the closenesses of the test vector to each training vector in a particular category. Category closenesses are coded by currents which feed into an “early bird” winner-take-all circuit on the third chip that selects the category closest to the test vector. Parzen classifiers offer superior classification accuracy than the common nearest neighbor Hamming networks. A high degree of parallelism allows for O(1) time complexity and the chips are tillable for increased training vector storage capacity. Proof-of-concept chips were fabricated through the MOSIS chip prototyping service and successfully tested View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Incremental communication for multilayer neural networks: error analysis

    Page(s): 68 - 82
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB)  

    Artificial neural networks (ANNs) involve a large amount of internode communications. To reduce the communication cost as well as the time of learning process in ANNs, we earlier proposed (1995) an incremental internode communication method. In the incremental communication method, instead of communicating the full magnitude of the output value of a node, only the increment or decrement to its previous value is sent to a communication link. In this paper, the effects of the limited precision incremental communication method on the convergence behavior and performance of multilayer neural networks are investigated. The nonlinear aspects of representing the incremental values with reduced (limited) precision for the commonly used error backpropagation training algorithm are analyzed. It is shown that the nonlinear effect of small perturbations in the input(s)/output of a node does not cause instability. The analysis is supported by simulation studies of two problems. The simulation results demonstrate that the limited precision errors are bounded and do not seriously affect the convergence of multilayer neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Limitations of nonlinear PCA as performed with generic neural networks

    Page(s): 165 - 173
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (276 KB)  

    Kramer's (1991) nonlinear principal components analysis (NLPCA) neural networks are feedforward autoassociative networks with five layers. The third layer has fewer nodes than the input or output layers. This paper proposes a geometric interpretation for Kramer's method by showing that NLPCA fits a lower-dimensional curve or surface through the training data. The first three layers project observations onto the curve or surface giving scores. The last three layers define the curve or surface. The first three layers are a continuous function, which we show has several implications: NLPCA “projections” are suboptimal producing larger approximation error, NLPCA is unable to model curves and surfaces that intersect themselves, and NLPCA cannot parameterize curves with parameterizations having discontinuous jumps. We establish results on the identification of score values and discuss their implications on interpreting score values. We discuss the relationship between NLPCA and principal curves and surfaces, another nonlinear feature extraction method View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning in certainty-factor-based multilayer neural networks for classification

    Page(s): 151 - 158
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (196 KB)  

    The computational framework of rule-based neural networks inherits from the neural network and the inference engine of an expert system. In one approach, the network activation function is based on the certainty factor (CF) model of MYCIN-like systems. In this paper, it is shown theoretically that the neural network using the CF-based activation function requires relatively small sample sizes for correct generalization. This result is also confirmed by empirical studies in several independent domains View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Doubly stochastic Poisson processes in artificial neural learning

    Page(s): 229 - 231
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (96 KB)  

    This paper investigates neuron activation statistics in artificial neural networks employing stochastic arithmetic. It is shown that a doubly stochastic Poisson process is an appropriate model for the signals in these circuits View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A discrete dynamics model for synchronization of pulse-coupled oscillators

    Page(s): 51 - 57
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (160 KB)  

    Biological information processing systems employ a variety of feature types. It has been postulated that oscillator synchronization is the mechanism for binding these features together to realize coherent perception. A discrete dynamic model of a coupled system of oscillators is presented. The network of oscillators converges to a state where subpopulations of cells become phase synchronized. It has potential applications to describing biological perception as well as for the construction of multifeature pattern recognition systems. It is shown that this model can be used to detect the presence of short line segments in the boundary contour of an object. The Hough transform, which is the standard method for detecting curve segments of a specified shape in an image was found not to be effective for this application. Implementation of the discrete dynamics model of oscillator synchronization is much easier than the differential equation models that have appeared in the literature. A systematic numerical investigation of the convergence properties of the model has been performed and it is shown that the discrete dynamics model can scale up to large number of oscillators View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Compensatory neurofuzzy systems with fast learning algorithms

    Page(s): 83 - 105
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (596 KB)  

    In this paper, a new adaptive fuzzy reasoning method using compensatory fuzzy operators is proposed to make a fuzzy logic system more adaptive and more effective. Such a compensatory fuzzy logic system is proved to be a universal approximator. The compensatory neural fuzzy networks built by both control-oriented fuzzy neurons and decision-oriented fuzzy neurons cannot only adaptively adjust fuzzy membership functions but also dynamically optimize the adaptive fuzzy reasoning by using a compensatory learning algorithm. The simulation results of a cart-pole balancing system and nonlinear system modeling have shown that: 1) the compensatory neurofuzzy system can effectively learn commonly used fuzzy IF-THEN rules from either well-defined initial data or ill-defined data; 2) the convergence speed of the compensatory learning algorithm is faster than that of the conventional backpropagation algorithm; and 3) the efficiency of the compensatory learning algorithm can be improved by choosing an appropriate compensatory degree View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fast training of recurrent networks based on the EM algorithm

    Page(s): 11 - 26
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (680 KB)  

    In this work, a probabilistic model is established for recurrent networks. The expectation-maximization (EM) algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean-field approximation. This new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons. These neurons are then trained via a linear weighted regression algorithm. The training time has been improved by five to 15 times on benchmark problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Long-term attraction in higher order neural networks

    Page(s): 42 - 50
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (468 KB)  

    Recent results on the memory storage capacity of higher order neural networks indicate a significant improvement compared to the limited capacity of the Hopfield model. However, such results have so far been obtained under the restriction that only a single iteration is allowed to converge. This paper presents a indirect convergence (long-term attraction) analysis of higher order neural networks. Our main result is that for any κd<d!2d-1/(2d)!, and 0⩽ρ<1/2, a Hebbian higher order neural network of order d with n neurons can store a random set of κdnd/log n fundamental memories such that almost all memories have an attraction radius of size ρn. If κd<d!2d-1/((2d)!(d+1)), then all memories possess this property simultaneously. It indicates that the lower bounds on the long-term attraction capacities are larger than the corresponding direct convergence capacities by a factor of 1/(1-2ρ) 2d. In addition we upper bound the convergence rate (number of iterations required to converge). This bound is asymptotically independent of n. Similar results are obtained for zero diagonal higher order neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Quantizing for minimum average misclassification risk

    Page(s): 174 - 182
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    In pattern classification, a decision rule is a labeled partition of the observation space, where labels represent classes. A way to establish a decision rule is to attach a label to each code vector of a vector quantizer (VQ). When a labeled VQ is adopted as a classifier, we have to design it in such a way that high classification performance is obtained by a given number of code vectors. In this paper we propose a learning algorithm which optimizes the position of labeled code vectors in the observation space under the minimum average misclassification risk criterion View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A smart pixel-based feedforward neural network

    Page(s): 159 - 164
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    A novel smart pixel-based neural network was realized experimentally. The matrix multiplication is split into positive and negative components and computed optically. The necessary subtraction, binarization, and transmission of the resulting matrices is accomplished via a prototype smart pixel spatial light modulator. The result is a neural network that performs truly parallel computation without requiring the use of an external processor View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive unsupervised extraction of one component of a linear mixture with a single neuron

    Page(s): 123 - 138
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (700 KB)  

    Extracting one specific component of a linear mixture is to isolate it due to the observation of several mixtures of all the components. This is done in an unsupervised way, based on the sole knowledge that the components are independent. The classical solution is independent component analysis which extracts the components all at the same time. In this paper, given at least as many sensors as components, we propose a simpler approach which independently extracts each component with one neuron. The weights of the neuron are optimized by minimizing an even polynomial of its output. The corresponding adaptive algorithm is an extended anti-Hebbian rule with very low complexity. It can extract any specific negative kurtosis component. Global stability of the algorithm is investigated as well as steady-state fluctuations. The influence of additive noise is also considered. These theoretical results are thoroughly confirmed by computer simulations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cross-validation with active pattern selection for neural-network classifiers

    Page(s): 35 - 41
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    We propose a new approach for leave-one-out cross-validation of neural-network classifiers called “cross-validation with active pattern selection” (CV/APS). In CV/APS, the contribution of the training patterns to network learning is estimated and this information is used for active selection of CV patterns. On the tested examples, the computational cost of CV can be drastically reduced with only small or no errors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on convergence under dynamical thresholds with delays

    Page(s): 231 - 233
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (64 KB)  

    We complement the study of the asymptotic behaviour of the dynamical threshold neuron model with delay, introduced by Gopalsamy and Leung (1997), by providing a description of the dynamics of the system in the remaining parameters range. We characterize the regions of “harmless” delays and those in which delay-induced oscillations appear View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global convergence of Oja's subspace algorithm for principal component extraction

    Page(s): 58 - 67
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (528 KB)  

    Oja's principal subspace algorithm is a well-known and powerful technique for learning and tracking principal information in time series. A thorough investigation of the convergence property of Oja's algorithm is undertaken in this paper. The asymptotic convergence rates of the algorithm is discovered. The dependence of the algorithm on its initial weight matrix and the singularity of the data covariance matrix is comprehensively addressed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Guaranteed two-pass convergence for supervised and inferential learning

    Page(s): 195 - 204
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (268 KB)  

    We present a theoretical analysis of a version of the LAPART adaptive inferencing neural network. Our main result is a proof that the new architecture, called LAPART 2, converges in two passes through a fixed training set of inputs. We also prove that it does not suffer from template proliferation. For comparison, Georgiopoulos et al. (1994) have proved the upper bound n-1 on the number of passes required for convergence for the ARTMAP architecture, where n is the size of the binary pattern input space. If the ARTMAP result is regarded as an n-pass, or finite-pass, convergence result, ours is then a two-pass, or fixed-pass, convergence result. Our results have added significance in that they apply to set-valued mappings, as opposed to the usual supervised learning model of affixing labels to classes View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiple descent cost competition: restorable self-organization and multimedia information processing

    Page(s): 106 - 122
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB)  

    Multiple descent cost competition is a composition of learning phases for minimizing a given measure of total performance, i.e., cost. In the first phase of descent cost learning, elements of source data are grouped. Simultaneously, a weight vector for minimal learning, (a winner), is found. Then, the winner and its partners are updated for further cost reduction. Therefore, two classes of self-organizing feature maps are generated: a grouping feature map, and an ordinary weight vector feature map. The grouping feature map, together with the winners, retains most of the source data information. This feature map is able to assist in a high quality approximation of the original data. In the paper, the total algorithm of the multiple descent cost competition is explained and image processing concepts are introduced. A still image is first data-compressed, then a restored image is morphed using the grouping feature map by receiving directions given by an external intelligence. Next, an interpolation of frames is applied in order to complete animation coding. Examples of multimedia processing on virtual digital movies are given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A direct adaptive neural-network control for unknown nonlinear systems and its application

    Page(s): 27 - 34
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    In this paper a direct adaptive neural-network control strategy for unknown nonlinear systems is presented. The system considered is described by an unknown NARMA model, and a feedforward neural network is used to learn the system. Taking the neural network as a neural model of the system, control signals are directly obtained by minimizing either the instant difference or the cumulative differences between a set point and the output of the neural model. Since the training algorithm guarantees that the output of the neural model approaches that of the actual system, it is shown that the control signals obtained can also make the real system output close to the set point. An application to a flow-rate control system is included to demonstrate the applicability of the proposed method and desired results are obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope