By Topic

Neural Networks, IEEE Transactions on

Issue 5 • Date Sept. 1999

Filter Results

Displaying Results 1 - 25 of 27
  • Guest editorial vapnik-chervonenkis (vc) learning theory and its applications

    Page(s): 985 - 987
    Save to Project icon | Request Permissions | PDF file iconPDF (21 KB)  
    Freely Available from IEEE
  • Fixed bit-rate image compression using a parallel-structure multilayer neural network

    Page(s): 1166 - 1172
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (140 KB)  

    Picture compression algorithms, using a parallel structure of neural networks, have recently been described. Although these algorithms are intrinsically robust, and may therefore be used in high noise environments, they suffer from several drawbacks: high computational complexity, moderate reconstructed picture qualities, and a variable bit-rate. In this paper, we describe a simple parallel structure in which all three drawbacks are eliminated: the computational complexity is low, the quality of the decompressed picture is high, and the bit-rate is fixed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Lagrangian network for kinematic control of redundant robot manipulators

    Page(s): 1123 - 1132
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (228 KB)  

    A recurrent neural network, called the Lagrangian network, is presented for the kinematic control of redundant robot manipulators. The optimal redundancy resolution is determined by the Lagrangian network through real-time solution to the inverse kinematics problem formulated as a quadratic optimization problem. While the signal for a desired velocity of the end-effector is fed into the inputs of the Lagrangian network, it generates the joint velocity vector of the manipulator in its outputs along with the associated Lagrange multipliers. The proposed Lagrangian network is shown to be capable of asymptotic tracking for the motion control of kinematically redundant manipulators View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural-network prediction with noisy predictors

    Page(s): 1196 - 1203
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (184 KB)  

    Very often the input variables for neural-network predictions contain measurement errors. In particular, this may happen because the original input variables are often not available at the time of prediction and have to be replaced by predicted values themselves. This issue is usually ignored and results in non-optimal predictions. This paper shows that under some general conditions, the optimal prediction using noisy input variables can be represented by a neural network with the same structure and the same weights as the optimal prediction using exact input variables. Only the activation functions have to be adjusted. Therefore, we can achieve optimal prediction without costly retraining of the neural network. We explicitly provide an exact formula for adjusting the activation functions in a logistic network with Gaussian measurement errors in input variables. This approach is illustrated by an application to short-term load forecasting View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal linear compression under unreliable representation and robust PCA neural models

    Page(s): 1186 - 1195
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (200 KB)  

    In a typical linear data compression system the representation variables resulting from the coding operation are assumed totally reliable and therefore the solution in the mean-squared-error sense is an orthogonal projector to the so-called principal component subspace. When the representation variables are contaminated by additive noise which is uncorrelated with the signal, the problem is called noisy principal component analysis (NPCA) and the optimal MSE solution is not a trivial extension of PCA. We show that: the problem is not well defined unless we impose explicit or implicit constraints on either the coding or the decoding operator; orthogonality is not a property of the optimal solution under most constraints; and the signal components may or may not be reconstructed depending on the noise level. As the noise power increases, we observe rank reduction in the optimal solution under most reasonable constraints. In these cases it appears that it is preferable to omit the smaller signal components rather than attempting to reconstruct them. Finally, we show that standard Hebbian-type PCA learning algorithms are not optimally robust to noise, and propose a new Hebbian-type learning algorithm which is optimally robust in the NPCA sense View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Simple and robust methods for support vector expansions

    Page(s): 1038 - 1047
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    Most support vector (SV) methods proposed in the recent literature can be viewed in a unified framework with great flexibility in terms of the choice of the kernel functions and their constraints. We show that all these problems can be solved within a unique approach if we are equipped with a robust method for finding a sparse solution of a linear system. Moreover, for such a purpose, we propose an iterative algorithm that can be simply implemented. Finally, we compare the classical SV approach with other, recently proposed, cross-correlation based, alternative methods. The simplicity of their implementation and the possibility of exactly calculating their computational complexity constitute important advantages in a real-time signal processing scenario View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Moderating the outputs of support vector machine classifiers

    Page(s): 1018 - 1031
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (316 KB)  

    In this paper, we extend the use of moderated outputs to the support vector machine (SVM) by making use of a relationship between SVM and the evidence framework. The moderated output is more in line with the Bayesian idea that the posterior weight distribution should be taken into account upon prediction, and it also alleviates the usual tendency of assigning overly high confidence to the estimated class memberships of the test patterns. Moreover, the moderated output derived here can be taken as an approximation to the posterior class probability. Hence, meaningful rejection thresholds can be assigned and outputs from several networks can be directly compared. Experimental results on both artificial and real-world data are also discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural-network models for the blood glucose metabolism of a diabetic

    Page(s): 1204 - 1213
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    We study the application of neural networks to modeling the blood glucose metabolism of a diabetic. In particular we consider recurrent neural networks and time series convolution neural networks which we compare to linear models and to nonlinear compartment models. We include a linear error model to take into account the uncertainty in the system and for handling missing blood glucose observations. Our results indicate that best performance can be achieved by the combination of the recurrent neural network and the linear error model View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks

    Page(s): 1239 - 1243
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (104 KB)  

    Presents a two-level learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimized using a genetic algorithm (GA) at the upper level. Nonlinear time series modeling and prediction is used as an example to demonstrate the effectiveness of this hierarchical learning approach View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Model complexity control for regression using VC generalization bounds

    Page(s): 1075 - 1089
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    It is well known that for a given sample size there exists a model of optimal complexity corresponding to the smallest prediction (generalization) error. Hence, any method for learning from finite samples needs to have some provisions for complexity control. Existing implementations of complexity control include penalization (or regularization), weight decay (in neural networks), and various greedy procedures (aka constructive, growing, or pruning methods). There are numerous proposals for determining optimal model complexity (aka model selection) based on various (asymptotic) analytic estimates of the prediction risk and on resampling approaches. Nonasymptotic bounds on the prediction risk based on Vapnik-Chervonenkis (VC)-theory have been proposed by Vapnik. This paper describes application of VC-bounds to regression problems with the usual squared loss. An empirical study is performed for settings where the VC-bounds can be rigorously applied, i.e., linear models and penalized linear models where the VC-dimension can be accurately estimated, and the empirical risk can be reliably minimized. Empirical comparisons between model selection using VC-bounds and classical methods are performed for various noise levels, sample size, target functions and types of approximating functions. Our results demonstrate the advantages of VC-based complexity control with finite samples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An axiomatic approach to soft learning vector quantization and clustering

    Page(s): 1153 - 1165
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (400 KB)  

    This paper presents an axiomatic approach to soft learning vector quantization (LVQ) and clustering based on reformulation. The reformulation of the fuzzy c-means (FCM) algorithm provides the basis for reformulating entropy-constrained fuzzy clustering (ECFC) algorithms. According to the proposed approach, the development of specific algorithms reduces to the selection of a generator function. Linear generator functions lead to the FCM and fuzzy learning vector quantization algorithms while exponential generator functions lead to ECFC and entropy-constrained learning vector quantization algorithms. The reformulation of LVQ and clustering algorithms also provides the basis for developing uncertainty measures that can identify feature vectors equidistant from all prototypes. These measures are employed by a procedure developed to make soft LVQ and clustering algorithms capable of identifying outliers in the data set. This procedure is evaluated by testing the algorithms generated by linear and exponential generator functions on speech data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Presupervised and post-supervised prototype classifier design

    Page(s): 1142 - 1152
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (292 KB)  

    We extend the nearest prototype classifier to a generalized nearest prototype classifier (GNPC). The GNPC uses “soft” labeling of the prototypes in the classes, thereby encompassing a variety of classifiers. Based on how the prototypes are found we distinguish between presupervised and post-supervised GNPC designs. We derive the conditions for optimality of two designs where prototypes represent: 1) the components of class-conditional mixture densities (presupervised design); or 2) the components of the unconditional mixture density (post-supervised design). An artificial data set and the “satimage” data set from the database ELENA are used to experimentally study the two approaches. A radial basis function network is used as a representative of each GNPC type. Neither the theoretical nor the experimental results indicate clear reasons to prefer one of the approaches. The post-supervised GNPC design tends to be more robust and less accurate than the presupervised one View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sub-symbolically managing pieces of symbolical functions for sorting

    Page(s): 1099 - 1122
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (544 KB)  

    We present a hybrid system for managing both symbolic and sub-symbolic knowledge in a uniform way. Our aim is to solve problems where some gap in formal theories occurs which stops one from getting a fully symbolical solution. The idea is to use neural modules to functionally connect pieces of symbolic knowledge, such as mathematical formulas and deductive rules. The whole system is trained through a backpropagation learning algorithm where all (symbolic or sub-symbolic) free parameters are updated piping back the error through each component of the system. The structure of this system is very general, possibly varying over time and managing fuzzy variables and decision trees. We use as a test-bed the problem of sorting a file, where suitable suggestions on next sorting moves are supplied by the network also on the basis of the hints provided by some conventional sorters. A comprehensive discussion of system performance is provided in order to understand behaviors and capabilities of the proposed hybrid system View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Successive overrelaxation for support vector machines

    Page(s): 1032 - 1037
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (156 KB)  

    Successive overrelaxation (SOR) for symmetric linear complementarity problems and quadratic programs is used to train a support vector machine (SVM) for discriminating between the elements of two massive datasets, each with millions of points. Because SOR handles one point at a time, similar to Platt's sequential minimal optimization (SMO) algorithm (1999) which handles two constraints at a time and Joachims' SVMlight (1998) which handles a small number of points at a time, SOR can process very large datasets that need not reside in memory. The algorithm converges linearly to a solution. Encouraging numerical results are presented on datasets with up to 10 000 000 points. Such massive discrimination problems cannot be processed by conventional linear or quadratic programming methods, and to our knowledge have not been solved by other methods. On smaller problems, SOR was faster than SVMlight and comparable or faster than SMO View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analog integrated circuits for the Lotka-Volterra competitive neural networks

    Page(s): 1222 - 1231
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (932 KB)  

    A subthreshold MOS integrated circuit (IC) is designed and fabricated for implementing a competitive neural network of the Lotka-Volterra (LV) type which is derived from conventional membrane dynamics of neurons and is used for the selection of external inputs. The steady-state solutions to the LV equation can be classified into three types, each of which represents qualitatively different selection behavior. Among the solutions, the winners-share-all (WSA) solution in which a certain number of neurons remain activated in steady states is particularly useful owing to robustness in the selection of inputs from a noisy environment. The measured results of the fabricated LV ICs agree well with the theoretical prediction as long as the influence of device mismatches is small. Furthermore, results of extensive circuit simulations prove that the large-scale LV circuit producing the WSA solution does exhibit a reliable selection compared with winner-take-all circuits, in the possible presence of device mismatches View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Task decomposition and module combination based on class relations: a modular neural network for pattern classification

    Page(s): 1244 - 1256
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (816 KB)  

    We propose a method for decomposing pattern classification problems based on the class relations among training data. By using this method, we can divide a K-class classification problem into a series of (2K) two-class problems. These two-class problems are to discriminate class Ci from class Cj for i=1, …, K and j=i+1, while the existence of the training data belonging to the other K-2 classes is ignored. If the two-class problem of discriminating class Ci from class Cj is still hard to be learned, we can further break down it into a set of two-class subproblems as small as we expect. Since each of the two-class problems can be treated as a completely separate classification problem with the proposed learning framework, all of the two-class problems can be learned in parallel. We also propose two module combination principles which give practical guidelines in integrating individual trained network modules. After learning of each of the two-class problems with a network module, we can easily integrate all of the trained modules into a min-max modular (M3) network according to the module combination principles and obtain a solution to the original problem. Consequently, a large-scale and complex K-class classification problem can be solved effortlessly and efficiently by learning a series of smaller and simpler two-class problems in parallel View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fusion of face and speech data for person identity verification

    Page(s): 1065 - 1074
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    Biometric person identity authentication is gaining more and more attention. The authentication task performed by an expert is a binary classification problem: reject or accept identity claim. Combining experts, each based on a different modality (speech, face, fingerprint, etc.), increases the performance and robustness of identity authentication systems. In this context, a key issue is the fusion of the different experts for taking a final decision (i.e., accept or reject identity claim). We propose to evaluate different binary classification schemes (support vector machine, multilayer perceptron, C4.5 decision tree, Fisher's linear discriminant, Bayesian classifier) to carry on the fusion. The experimental results show that support vector machines and Bayesian classifier achieve almost the same performances, and both outperform the other evaluated classifiers View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Support vector machines for histogram-based image classification

    Page(s): 1055 - 1064
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (676 KB)  

    Traditional classification approaches generalize poorly on image classification tasks, because of the high dimensionality of the feature space. This paper shows that support vector machines (SVM) can generalize well on difficult image classification problems where the only features are high dimensional histograms. Heavy-tailed RBF kernels of the form K(x, y)=eΣi|xia-yia|b with a ⩽1 and b⩽2 are evaluated on the classification of images extracted from the Corel stock photo collection and shown to far outperform traditional polynomial or Gaussian radial basis function (RBF) kernels. Moreover, we observed that a simple remapping of the input xi→xia improves the performance of linear SVM to such an extend that it makes them, for this problem, a valid alternative to RBF kernels View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Input space versus feature space in kernel-based methods

    Page(s): 1000 - 1017
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1216 KB)  

    This paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector (SV) kernel functions. We first discuss the geometry of feature space. In particular, we review what is known about the shape of the image of input space under the feature space map, and how this influences the capacity of SV methods. Following this, we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel, using the example of the class of inhomogeneous polynomial kernels, which are often used in SV pattern recognition. We then discuss the connection between feature space and input space by dealing with the question of how one can, given some vector in feature space, find a preimage (exact or approximate) in input space. We describe algorithms to tackle this issue, and show their utility in two applications of kernel methods. First, we use it to reduce the computational complexity of SV decision functions; second, we combine it with the kernel PCA algorithm, thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real-world data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust backpropagation training algorithm for multilayered neural tracking controller

    Page(s): 1133 - 1141
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    A robust backpropagation training algorithm with a dead zone scheme is used for the online tuning of the neural-network (NN) tracking control system. This assures the convergence of the multilayer NN in the presence of disturbance. It is proved in this paper that the selection of a smaller range of the dead zone leads to a smaller estimate error of the NN, and hence a smaller tracking error of the NN tracking controller. The proposed algorithm is applied to a three-layered network with adjustable weights and a complete convergence proof is provided. The results can also be extended to the network with more hidden layers View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Dynamic range and sensitivity adaptation in a silicon spiking neuron

    Page(s): 1232 - 1238
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (148 KB)  

    We propose an adaptive procedure that enables a spiking neuron, whether artificial or biological, to make optimal use of its dynamic range and gain. We discuss an analog electronic circuit implementation of this algorithm using a biologically realistic artificial “silicon” neuron. The adaptation procedure adapts the neuron's firing threshold and the sensitivity (or gain) of its current-frequency relationship to match the DC offset (or mean) and the dynamic range (or variance) of the time-varying somatic input current. The neuron extracts the minimum and maximum levels of the reconstructed somatic current signals from the cell's own spike trains. These are used to regulate the somatic leak conductance in order to shift the somatic current-frequency relation and to adjust a calcium-activated potassium conductance to change the dynamic range of the cell's somatic current-frequency relationship. We report experimental data from a test neuron-built using analog subthreshold CMOS VLSI technology-that shows the expected behavior View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A novel approach to fault diagnosis in multicircuit transmission lines using fuzzy ARTmap neural networks

    Page(s): 1214 - 1221
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (152 KB)  

    Addresses the problems of fault diagnosis in complex multicircuit transmission systems, in particular those arising due to mutual coupling between the two parallel circuits under different fault conditions; the problems are compounded by the fact that this mutual coupling is highly variable in nature. In this respect, artificial intelligence (AI) techniques provide the ability to classify the faulted phase/phases by identifying different patterns of the associated voltages and currents. A fuzzy ARTmap (adaptive resonance theory) neural network is employed and is found to be well-suited for solving the complex fault classification problem under various system and fault conditions. Emphasis is placed on introducing the background of AI techniques as applied to the specific problem, followed by a description of the methodology adopted for training the fuzzy ARTmap neural network, which is proving to be a very useful and powerful tool for power system engineers. Furthermore, this classification technique is compared with a neural network technique based on the error backpropagation training algorithm, and it is shown that the former technique is better suited for solving the fault diagnosis problem in complex multicircuit transmission systems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An overview of statistical learning theory

    Page(s): 988 - 999
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    Statistical learning theory was introduced in the late 1960's. Until the 1990's it was a purely theoretical analysis of the problem of function estimation from a given collection of data. In the middle of the 1990's new types of learning algorithms (called support vector machines) based on the developed theory were proposed. This made statistical learning theory not only a tool for the theoretical analysis but also a tool for creating practical algorithms for estimating multidimensional functions. This article presents a very general overview of statistical learning theory including both theoretical and algorithmic aspects of the theory. The goal of this overview is to demonstrate how the abstract learning theory established conditions for generalization which are more general than those discussed in classical statistical paradigms and how the understanding of these conditions inspired new algorithmic approaches to function estimation problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A transiently chaotic neural-network implementation of the CDMA multiuser detector

    Page(s): 1257 - 1259
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (112 KB)  

    The complex dynamics of the chaotic neural networks makes it possible for them to escape from local minimum of the simple gradient descent neurodynamics. We use a transiently chaotic neural network to detect the CDMA multiuser signals and hence obtain an implementation scheme of the CDMA multiuser detector (TCNN-MD). Computer simulation results show that the proposed detector is clearly superior to Hopfield neural-network-based detector View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Blind separation of uniformly distributed signals: a general approach

    Page(s): 1173 - 1185
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (312 KB)  

    A general algorithm for blind separation of uniformly distributed signals is presented. First, maximum likelihood equations are obtained for dealing with this task. It is difficult to obtain a closed form maximum likelihood solution for arbitrary mixing matrix. The learning rules are obtained based on the geometric interpretation of the maximum likelihood estimator. The algorithm, under special constraint of orthogonal mixing matrix, is the same as the O(1/T2) convergent algorithm. Special noise correction mechanisms are incorporated in the algorithm, and it has been found that the algorithm exhibits stable performance even in the presence of large amount of noise View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope