By Topic

Neural Networks, IEEE Transactions on

Issue 1 • Date Jan. 1998

Filter Results

Displaying Results 1 - 25 of 27
  • Elements Of Artificial Neural Networks [Book Reviews]

    Publication Year: 1998 , Page(s): 234 - 235
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | PDF file iconPDF (17 KB)  
    Freely Available from IEEE
  • Neural Network Analysis, Architectures And Applications [Books in Brief]

    Publication Year: 1998 , Page(s): 236
    Save to Project icon | Request Permissions | PDF file iconPDF (6 KB)  
    Freely Available from IEEE
  • A smart pixel-based feedforward neural network

    Publication Year: 1998 , Page(s): 159 - 164
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    A novel smart pixel-based neural network was realized experimentally. The matrix multiplication is split into positive and negative components and computed optically. The necessary subtraction, binarization, and transmission of the resulting matrices is accomplished via a prototype smart pixel spatial light modulator. The result is a neural network that performs truly parallel computation without requiring the use of an external processor View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A bootstrap evaluation of the effect of data splitting on financial time series

    Publication Year: 1998 , Page(s): 213 - 220
    Cited by:  Papers (15)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (172 KB)  

    Exposes problems of the commonly used technique of splitting the available data into training, validation, and test sets that are held fixed, warns about drawing too strong conclusions from such static splits, and shows potential pitfalls of ignoring variability across splits. Using a bootstrap or resampling method, we compare the uncertainty in the solution stemming from the data splitting with neural-network specific uncertainties (parameter initialization, choice of number of hidden units, etc.). We present two results on data from the New York Stock Exchange. First, the variation due to different resamplings is significantly larger than the variation due to different network conditions. This result implies that it is important to not over-interpret a model (or an ensemble of models) estimated on one specific split of the data. Second, on each split, the neural-network solution with early stopping is very close to a linear model; no significant nonlinearities are extracted View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Incremental communication for multilayer neural networks: error analysis

    Publication Year: 1998 , Page(s): 68 - 82
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB)  

    Artificial neural networks (ANNs) involve a large amount of internode communications. To reduce the communication cost as well as the time of learning process in ANNs, we earlier proposed (1995) an incremental internode communication method. In the incremental communication method, instead of communicating the full magnitude of the output value of a node, only the increment or decrement to its previous value is sent to a communication link. In this paper, the effects of the limited precision incremental communication method on the convergence behavior and performance of multilayer neural networks are investigated. The nonlinear aspects of representing the incremental values with reduced (limited) precision for the commonly used error backpropagation training algorithm are analyzed. It is shown that the nonlinear effect of small perturbations in the input(s)/output of a node does not cause instability. The analysis is supported by simulation studies of two problems. The simulation results demonstrate that the limited precision errors are bounded and do not seriously affect the convergence of multilayer neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A discrete dynamics model for synchronization of pulse-coupled oscillators

    Publication Year: 1998 , Page(s): 51 - 57
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (160 KB)  

    Biological information processing systems employ a variety of feature types. It has been postulated that oscillator synchronization is the mechanism for binding these features together to realize coherent perception. A discrete dynamic model of a coupled system of oscillators is presented. The network of oscillators converges to a state where subpopulations of cells become phase synchronized. It has potential applications to describing biological perception as well as for the construction of multifeature pattern recognition systems. It is shown that this model can be used to detect the presence of short line segments in the boundary contour of an object. The Hough transform, which is the standard method for detecting curve segments of a specified shape in an image was found not to be effective for this application. Implementation of the discrete dynamics model of oscillator synchronization is much easier than the differential equation models that have appeared in the literature. A systematic numerical investigation of the convergence properties of the model has been performed and it is shown that the discrete dynamics model can scale up to large number of oscillators View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions

    Publication Year: 1998 , Page(s): 224 - 229
    Cited by:  Papers (85)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (220 KB)  

    It is well known that standard single-hidden layer feedforward networks (SLFNs) with at most N hidden neurons (including biases) can learn N distinct samples (xi,ti) with zero error, and the weights connecting the input neurons and the hidden neurons can be chosen “almost” arbitrarily. However, these results have been obtained for the case when the activation function for the hidden neurons is the signum function. This paper rigorously proves that standard single-hidden layer feedforward networks (SLFNs) with at most N hidden neurons and with any bounded nonlinear activation function which has a limit at one infinity can learn N distinct samples (xi,ti) with zero error. The previous method of arbitrarily choosing weights is not feasible for any SLFN. The proof of our result is constructive and thus gives a method to directly find the weights of the standard SLFNs with any such bounded nonlinear activation function as opposed to iterative training algorithms in the literature View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability analysis for neural dynamics with time-varying delays

    Publication Year: 1998 , Page(s): 221 - 223
    Cited by:  Papers (52)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    By using the usual additive neural-network model, a delay-independent stability criterion for neural dynamics with perturbations of time-varying delays is derived. We extend previously known results obtained by Gopalsamy and He (1994) to the time varying delay case, and present decay estimates of solutions of neural networks. The asymptotic stability is global in the state space of neuronal activations. From the techniques used in this paper, it is shown that our criterion ensures stability of neural dynamics even when the delay functions vary violently with time. Our approach provides an effective method for the stability analysis of neural dynamics with delays View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Compensatory neurofuzzy systems with fast learning algorithms

    Publication Year: 1998 , Page(s): 83 - 105
    Cited by:  Papers (66)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (596 KB)  

    In this paper, a new adaptive fuzzy reasoning method using compensatory fuzzy operators is proposed to make a fuzzy logic system more adaptive and more effective. Such a compensatory fuzzy logic system is proved to be a universal approximator. The compensatory neural fuzzy networks built by both control-oriented fuzzy neurons and decision-oriented fuzzy neurons cannot only adaptively adjust fuzzy membership functions but also dynamically optimize the adaptive fuzzy reasoning by using a compensatory learning algorithm. The simulation results of a cart-pole balancing system and nonlinear system modeling have shown that: 1) the compensatory neurofuzzy system can effectively learn commonly used fuzzy IF-THEN rules from either well-defined initial data or ill-defined data; 2) the convergence speed of the compensatory learning algorithm is faster than that of the conventional backpropagation algorithm; and 3) the efficiency of the compensatory learning algorithm can be improved by choosing an appropriate compensatory degree View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A direct adaptive neural-network control for unknown nonlinear systems and its application

    Publication Year: 1998 , Page(s): 27 - 34
    Cited by:  Papers (87)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    In this paper a direct adaptive neural-network control strategy for unknown nonlinear systems is presented. The system considered is described by an unknown NARMA model, and a feedforward neural network is used to learn the system. Taking the neural network as a neural model of the system, control signals are directly obtained by minimizing either the instant difference or the cumulative differences between a set point and the output of the neural model. Since the training algorithm guarantees that the output of the neural model approaches that of the actual system, it is shown that the control signals obtained can also make the real system output close to the set point. An application to a flow-rate control system is included to demonstrate the applicability of the proposed method and desired results are obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Long-term attraction in higher order neural networks

    Publication Year: 1998 , Page(s): 42 - 50
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (468 KB)  

    Recent results on the memory storage capacity of higher order neural networks indicate a significant improvement compared to the limited capacity of the Hopfield model. However, such results have so far been obtained under the restriction that only a single iteration is allowed to converge. This paper presents a indirect convergence (long-term attraction) analysis of higher order neural networks. Our main result is that for any κd<d!2d-1/(2d)!, and 0⩽ρ<1/2, a Hebbian higher order neural network of order d with n neurons can store a random set of κdnd/log n fundamental memories such that almost all memories have an attraction radius of size ρn. If κd<d!2d-1/((2d)!(d+1)), then all memories possess this property simultaneously. It indicates that the lower bounds on the long-term attraction capacities are larger than the corresponding direct convergence capacities by a factor of 1/(1-2ρ) 2d. In addition we upper bound the convergence rate (number of iterations required to converge). This bound is asymptotically independent of n. Similar results are obtained for zero diagonal higher order neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Doubly stochastic Poisson processes in artificial neural learning

    Publication Year: 1998 , Page(s): 229 - 231
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (96 KB)  

    This paper investigates neuron activation statistics in artificial neural networks employing stochastic arithmetic. It is shown that a doubly stochastic Poisson process is an appropriate model for the signals in these circuits View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Quantizing for minimum average misclassification risk

    Publication Year: 1998 , Page(s): 174 - 182
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    In pattern classification, a decision rule is a labeled partition of the observation space, where labels represent classes. A way to establish a decision rule is to attach a label to each code vector of a vector quantizer (VQ). When a labeled VQ is adopted as a classifier, we have to design it in such a way that high classification performance is obtained by a given number of code vectors. In this paper we propose a learning algorithm which optimizes the position of labeled code vectors in the observation space under the minimum average misclassification risk criterion View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Guaranteed two-pass convergence for supervised and inferential learning

    Publication Year: 1998 , Page(s): 195 - 204
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (268 KB)  

    We present a theoretical analysis of a version of the LAPART adaptive inferencing neural network. Our main result is a proof that the new architecture, called LAPART 2, converges in two passes through a fixed training set of inputs. We also prove that it does not suffer from template proliferation. For comparison, Georgiopoulos et al. (1994) have proved the upper bound n-1 on the number of passes required for convergence for the ARTMAP architecture, where n is the size of the binary pattern input space. If the ARTMAP result is regarded as an n-pass, or finite-pass, convergence result, ours is then a two-pass, or fixed-pass, convergence result. Our results have added significance in that they apply to set-valued mappings, as opposed to the usual supervised learning model of affixing labels to classes View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiple descent cost competition: restorable self-organization and multimedia information processing

    Publication Year: 1998 , Page(s): 106 - 122
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB)  

    Multiple descent cost competition is a composition of learning phases for minimizing a given measure of total performance, i.e., cost. In the first phase of descent cost learning, elements of source data are grouped. Simultaneously, a weight vector for minimal learning, (a winner), is found. Then, the winner and its partners are updated for further cost reduction. Therefore, two classes of self-organizing feature maps are generated: a grouping feature map, and an ordinary weight vector feature map. The grouping feature map, together with the winners, retains most of the source data information. This feature map is able to assist in a high quality approximation of the original data. In the paper, the total algorithm of the multiple descent cost competition is explained and image processing concepts are introduced. A still image is first data-compressed, then a restored image is morphed using the grouping feature map by receiving directions given by an external intelligence. Next, an interpolation of frames is applied in order to complete animation coding. Examples of multimedia processing on virtual digital movies are given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cross-validation with active pattern selection for neural-network classifiers

    Publication Year: 1998 , Page(s): 35 - 41
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    We propose a new approach for leave-one-out cross-validation of neural-network classifiers called “cross-validation with active pattern selection” (CV/APS). In CV/APS, the contribution of the training patterns to network learning is estimated and this information is used for active selection of CV patterns. On the tested examples, the computational cost of CV can be drastically reduced with only small or no errors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning in certainty-factor-based multilayer neural networks for classification

    Publication Year: 1998 , Page(s): 151 - 158
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (196 KB)  

    The computational framework of rule-based neural networks inherits from the neural network and the inference engine of an expert system. In one approach, the network activation function is based on the certainty factor (CF) model of MYCIN-like systems. In this paper, it is shown theoretically that the neural network using the CF-based activation function requires relatively small sample sizes for correct generalization. This result is also confirmed by empirical studies in several independent domains View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on convergence under dynamical thresholds with delays

    Publication Year: 1998 , Page(s): 231 - 233
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (64 KB)  

    We complement the study of the asymptotic behaviour of the dynamical threshold neuron model with delay, introduced by Gopalsamy and Leung (1997), by providing a description of the dynamics of the system in the remaining parameters range. We characterize the regions of “harmless” delays and those in which delay-induced oscillations appear View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fast training of recurrent networks based on the EM algorithm

    Publication Year: 1998 , Page(s): 11 - 26
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (680 KB)  

    In this work, a probabilistic model is established for recurrent networks. The expectation-maximization (EM) algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean-field approximation. This new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons. These neurons are then trained via a linear weighted regression algorithm. The training time has been improved by five to 15 times on benchmark problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Limitations of nonlinear PCA as performed with generic neural networks

    Publication Year: 1998 , Page(s): 165 - 173
    Cited by:  Papers (39)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (276 KB)  

    Kramer's (1991) nonlinear principal components analysis (NLPCA) neural networks are feedforward autoassociative networks with five layers. The third layer has fewer nodes than the input or output layers. This paper proposes a geometric interpretation for Kramer's method by showing that NLPCA fits a lower-dimensional curve or surface through the training data. The first three layers project observations onto the curve or surface giving scores. The last three layers define the curve or surface. The first three layers are a continuous function, which we show has several implications: NLPCA “projections” are suboptimal producing larger approximation error, NLPCA is unable to model curves and surfaces that intersect themselves, and NLPCA cannot parameterize curves with parameterizations having discontinuous jumps. We establish results on the identification of score values and discuss their implications on interpreting score values. We discuss the relationship between NLPCA and principal curves and surfaces, another nonlinear feature extraction method View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis and design of primal-dual assignment networks

    Publication Year: 1998 , Page(s): 183 - 194
    Cited by:  Papers (9)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (456 KB)  

    The assignment problem is an archetypical combinatorial optimization problem having widespread applications. This paper presents two recurrent neural networks, a continuous-time one and a discrete-time one, for solving the assignment problem. Because the proposed recurrent neural networks solve the primal and dual assignment problems simultaneously, they are called primal-dual assignment networks. The primal-dual assignment networks are guaranteed to make optimal assignment regardless of initial conditions. Unlike the primal or dual assignment network, there is no time-varying design parameter in the primal-dual assignment networks. Therefore, they are more suitable for hardware implementation. The performance and operating characteristics of the primal-dual assignment networks are demonstrated by means of illustrative examples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A CMOS binary pattern classifier based on Parzen's method

    Publication Year: 1998 , Page(s): 2 - 10
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (352 KB)  

    Biological circuitry in the brain that has been associated with the Parzen method of classification inspired an analog CMOS binary pattern classifier. The circuitry resides on three separate chips. The first chip computes the closeness of a test vector to each training vector stored on the chip where “vector closeness” is defined as the number of bits two vectors have in common above some thresholds. The second chip computes the closeness of the test vector to each possible category where “category closeness” is defined as the sum of the closenesses of the test vector to each training vector in a particular category. Category closenesses are coded by currents which feed into an “early bird” winner-take-all circuit on the third chip that selects the category closest to the test vector. Parzen classifiers offer superior classification accuracy than the common nearest neighbor Hamming networks. A high degree of parallelism allows for O(1) time complexity and the chips are tillable for increased training vector storage capacity. Proof-of-concept chips were fabricated through the MOSIS chip prototyping service and successfully tested View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimization Techniques [Book in Brief]

    Publication Year: 1998 , Page(s): 236
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (8 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Global convergence of Oja's subspace algorithm for principal component extraction

    Publication Year: 1998 , Page(s): 58 - 67
    Cited by:  Papers (29)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (528 KB)  

    Oja's principal subspace algorithm is a well-known and powerful technique for learning and tracking principal information in time series. A thorough investigation of the convergence property of Oja's algorithm is undertaken in this paper. The asymptotic convergence rates of the algorithm is discovered. The dependence of the algorithm on its initial weight matrix and the singularity of the data covariance matrix is comprehensively addressed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • MART: a multichannel ART-based neural network

    Publication Year: 1998 , Page(s): 139 - 150
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (312 KB)  

    This paper describes MART, an ART-based neural network for adaptive classification of multichannel signal patterns without prior supervised learning. Like other ART-based classifiers, MART is especially suitable for situations in which not even the number of pattern categories to be distinguished is known a priori; its novelty lies in its truly multichannel orientation, especially its ability to quantify and take into account during pattern classification the different changing reliability of the individual signal channels. The extent to which this ability can reduce the creation of spurious or duplicate categories (a major problem for ART-based classifiers of noisy signals) is illustrated by evaluation of its performance in classifying QRS complexes in two-channel ECG traces which were taken from the MIT-BIH database and contaminated with noise View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope