Scheduled System Maintenance:
On May 6th, single article purchases and IEEE account management will be unavailable from 8:00 AM - 5:00 PM ET (12:00 - 21:00 UTC). We apologize for the inconvenience.
By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 1999

Filter Results

Displaying Results 1 - 25 of 32
  • Corrections to "a new error function at hidden layers for fast training of multilayer perceptrons"

    Publication Year: 1999 , Page(s): 1541
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (84 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author index

    Publication Year: 1999 , Page(s): 1 - 6
    Save to Project icon | Request Permissions | PDF file iconPDF (216 KB)  
    Freely Available from IEEE
  • Subject index

    Publication Year: 1999 , Page(s): 6 - 20
    Save to Project icon | Request Permissions | PDF file iconPDF (231 KB)  
    Freely Available from IEEE
  • Partition-based and sharp uniform error bounds

    Publication Year: 1999 , Page(s): 1315 - 1320
    Cited by:  Papers (2)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (128 KB)  

    This paper develops probabilistic bounds on out-of-sample error rates for several classifiers using a single set of in-sample data. The bounds are based on probabilities over partitions of the union of in-sample and out-of-sample data into in-sample and out-of-sample data sets, The bounds apply when in-sample and out-of-sample data are drawn from the same distribution. Partition-based bounds are stronger than the Vapnik-Chervonenkis bounds, but they require more computation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiple neural-network-based adaptive controller using orthonormal activation function neural networks

    Publication Year: 1999 , Page(s): 1494 - 1501
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (212 KB)  

    A direct adaptive control scheme is developed using orthonormal activation function-based neural networks (OAFNNs) for trajectory tracking control of a class of nonlinear systems. Multiple OAFNNs are employed in these controllers for feedforward compensation of unknown system dynamics. Choice of multiple OAFNNs allows a reduction in overall network size reducing the computational requirements. The network weights are tuned online, in real time. The overall stability of the system and the neural networks is guaranteed using Lyapunov analysis. The developed neural controllers are evaluated experimentally and the experimental results are shown to support theoretical analysis. The effects of network parameters on system performance are experimentally evaluated and are presented. The superior learning capability of OAFNNs is demonstrated through experimental results. The OAFNNs were able to model the true nature of the nonlinear system dynamics characteristics for a rolling-sliding contact as well as for stiction View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Design quality and robustness with neural networks

    Publication Year: 1999 , Page(s): 1518 - 1527
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (180 KB)  

    Data sets that are gathered for industrial applications are frequently noisy and present challenges that necessitate use of different methodologies. We present a successful example of generating concise and accurate neural-network models for multiple quality characteristics in injection molding. These models map process measurements to product quality. They are used for product and process design, addressing material and processing issues, and defining the operating window and its robustness. The models are developed based on data from designed and other experiments. Linear regression, decision tree induction, nonlinear regression, as well as “stepwise neural networks” were used for feature selection and model comparison. The final model consists of a neural network with three inputs, one hidden layer and five outputs, modeling five critical to quality variables simultaneously with high accuracy. The neural network was visualized for validation and insight View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic behavior of irreducible excitatory networks of analog graded-response neurons

    Publication Year: 1999 , Page(s): 1375 - 1381
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (220 KB)  

    In irreducible excitatory networks of analog graded-response neurons, the trajectories of most solutions tend to the equilibria. We derive sufficient conditions for such networks to be globally asymptotically stable. When the network possesses several locally stable equilibria, their location in the phase space is discussed and a description of their attraction basin is given. The results hold even when interunit transmission is delayed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The “weight smoothing” regularization of MLP for Jacobian stabilization

    Publication Year: 1999 , Page(s): 1502 - 1510
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (208 KB)  

    In an approximation problem with a neural network, a low-output root mean square error is not always a universal criterion. We investigate problems where the Jacobians-first derivative of an output value with respect to an input value-of the approximation model are needed and propose to add a quality criterion on these Jacobians during the learning step. More specifically, we focus on the approximation of functionals 𝒜, from a space of continuous functions (discretized in practice) to a scalar space. In this case, the approximation is confronted with the compensation phenomenon: a lower contribution of one input can be compensated by a larger one of its neighboring inputs. In this case, profiles (with respect to the input index) of neural Jacobians are very irregular instead of smooth. Then, the approximation of 𝒜 becomes an ill-posed problem because many solutions can be chosen by the learning process. We propose to introduce the smoothness of Jacobian profiles as an a priori information via a regularization technique and develop a new and efficient learning algorithm, called “weight smoothing”. We assess the robustness of the weight smoothing algorithm by testing it on a real and complex problem stemming from meteorology: the neural approximation of the forward model of radiative transfer equation in the atmosphere. The stabilized Jacobians of this model are then used in an inversion process to illustrate the improvement of the Jacobians after weight smoothing View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The application of neural networks to the papermaking industry

    Publication Year: 1999 , Page(s): 1456 - 1464
    Cited by:  Papers (16)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (128 KB)  

    This paper describes the application of neural network techniques to the papermaking industry, particularly for the prediction of paper “curl”. Paper curl is an important quality measure that can only be measured reliably off-line after manufacture, making it difficult to control. Here, we predict, before paper manufacture from characteristics of the current reel, whether the paper curl will be acceptable and the level of curl. For both issues the case of predicting the probability that paper will be “out-of-specification” and that of predicting the level of curl, we include confidence intervals indicating to the machine operator whether the predictions should be trusted. The results and the associated discussion describe a successful application of neural networks to a difficult, but important, real-world task taken from the papermaking industry. In addition the techniques described are widely applicable to industry where direct prediction of a quality measure and its acceptability are desirable View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Relative loss bounds for single neurons

    Publication Year: 1999 , Page(s): 1291 - 1304
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (348 KB)  

    We analyze and compare the well-known gradient descent algorithm and the more recent exponentiated gradient algorithm for training a single neuron with an arbitrary transfer function. Both algorithms are easily generalized to larger neural networks, and the generalization of gradient descent is the standard backpropagation algorithm. We prove worst-case loss bounds for both algorithms in the single neuron case. Since local minima make it difficult to prove worst case bounds for gradient-based algorithms, we must use a loss function that prevents the formation of spurious local minima. We define such a matching loss function for any strictly increasing differentiable transfer function and prove worst-case loss bounds for any such transfer function and its corresponding matching loss. The different forms of the two algorithms' bounds indicates that exponentiated gradient outperforms gradient descent when the inputs contain a large number of irrelevant components. Simulations on synthetic data confirm these analytical results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Training neural networks with additive noise in the desired signal

    Publication Year: 1999 , Page(s): 1511 - 1517
    Cited by:  Papers (15)  |  Patents (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (116 KB)  

    A global optimization strategy for training adaptive systems such as neural networks and adaptive filters (finite or infinite impulse response) is proposed. Instead of adding random noise to the weights as proposed in the past, additive random noise is injected directly into the desired signal. Experimental results show that this procedure also speeds up greatly the backpropagation algorithm. The method is very easy to implement in practice, preserving the backpropagation algorithm and requiring a single random generator with a monotonically decreasing step size per output channel. Hence, this is an ideal strategy to speed up supervised learning, and avoid local minima entrapment when the noise variance is appropriately scheduled View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Toward an optimal PRNN-based nonlinear predictor

    Publication Year: 1999 , Page(s): 1435 - 1442
    Cited by:  Papers (17)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (176 KB)  

    We present an approach for selecting optimal parameters for the pipelined recurrent neural network (PRNN) in the paradigm of nonlinear and nonstationary signal prediction. We consider the role of nesting, which is inherent to the PRNN architecture. The corresponding number of nested modules needed for a certain prediction task, and their contribution toward the final prediction gain give a thorough insight into the way the PRNN performs, and offers solutions for optimization of its parameters. In particular, nesting allows the forgetting factor in the cost function of the PRNN to exceed unity, hence it becomes an emphasis factor. This compensates for the small contribution of the distant modules to the prediction process, due to nesting, and helps to circumvent the problem of vanishing gradient, experienced in RNNs for prediction. The PRNN is shown to outperform the linear least mean square and recursive least squares predictors, as well as previously proposed PRNN schemes, at no expense of additional computational complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Complex cell prototype representation for face recognition

    Publication Year: 1999 , Page(s): 1528 - 1531
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    We propose a face recognition system based on a biologically inspired filtering method. Our work differs from previous proposals in: 1) the multistage filtering method employed; 2) the pyramid structure used, and most importantly; 3) the prototype construction scheme to determine the models stored in memory. The method is much simpler than previous proposals and relatively inexpensive computationally, while attaining error rates as low as 5%, very close to the best reported results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sample selection via clustering to construct support vector-like classifiers

    Publication Year: 1999 , Page(s): 1474 - 1481
    Cited by:  Papers (22)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    Explores the possibility of constructing RBF classifiers which, somewhat like support vector machines, use a reduced number of samples as centroids, by means of selecting samples in a direct way. Because sample selection is viewed as a hard computational problem, this selection is done after a previous vector quantization: this way obtains also other similar machines using centroids selected from those that are learned in a supervised manner. Several forms of designing these machines are considered, in particular with respect to sample selection; as well as some different criteria to train them. Simulation results for well-known classification problems show very good performance of the corresponding designs, improving that of support vector machines and reducing substantially their number of units. This shows that our interest in selecting samples (or centroids) in an efficient manner is justified. Many new research avenues appear from these experiments and discussions, as suggested in our conclusions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability analysis of Hopfield-type neural networks

    Publication Year: 1999 , Page(s): 1366 - 1374
    Cited by:  Papers (19)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (208 KB)  

    The paper applies several concepts in robust control research such as linear matrix inequalities, edge theorem, parameter-dependent Lyapunov function, and Popov criteria to investigate the stability property of Hopfield-type neural networks. The existence and uniqueness of an equilibrium is formulated as a matrix determinant problem. An induction scheme is used to find the equilibrium. To verify whether the determinant is nonzero for a class of matrix, a numerical range test is proposed. Several robust control techniques in particular linear matrix inequalities are used to characterize the local stability of the neural networks around the equilibrium. The global stability of the Hopfield neural networks is then addressed using a parameter-dependent Lyapunov function technique. All these results are shown to generalize existing results in verifying the existence/uniqueness of the equilibrium and local/global stability of Hopfield-type neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the implementation of frontier-to-root tree automata in recursive neural networks

    Publication Year: 1999 , Page(s): 1305 - 1314
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    We explore the node complexity of recursive neural network implementations of frontier-to-root tree automata (FRA). Specifically, we show that an FRAO (Mealy version) with m states, l input-output labels, and maximum rank N can be implemented by a recursive neural network with O(√(log l+log m)lmN/log l+N log m) units and four computational layers, i.e., without counting the input layer. A lower bound is derived which is tight when no restrictions are placed on the number of layers. Moreover, we present a construction with three computational layers having node complexity of O((log l+log m)√lm N) and O((log l+log m)lmN) connections. A construction with two computational layers is given that implements any given FRAO with a node complexity of O(lmN) and O((log l+log m)lmN) connections. As a corollary we also get a new upper bound for the implementation of finite-state automata into recurrent neural networks with three computational layers View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bayesian approach to neural-network modeling with input uncertainty

    Publication Year: 1999 , Page(s): 1261 - 1270
    Cited by:  Papers (14)  |  Patents (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (208 KB)  

    It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise or corruption. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural-network framework which allows for input noise provided that some model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method gives an additional term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable and sampling this jointly with the network weights using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • ANN-DT: an algorithm for extraction of decision trees from artificial neural networks

    Publication Year: 1999 , Page(s): 1392 - 1401
    Cited by:  Papers (27)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (176 KB)  

    Although artificial neural networks can represent a variety of complex systems with a high degree of accuracy, these connectionist models are difficult to interpret. This significantly limits the applicability of neural networks in practice, especially where a premium is placed on the comprehensibility or reliability of systems. A novel artificial neural-network decision tree algorithm (ANN-DT) is therefore proposed, which extracts binary decision trees from a trained neural network. The ANN-DT algorithm uses the neural network to generate outputs for samples interpolated from the training data set. In contrast to existing techniques, ANN-DT can extract rules from feedforward neural networks with continuous outputs. These rules are extracted from the neural network without making assumptions about the internal structure of the neural network or the features of the data. A novel attribute selection criterion based on a significance analysis of the variables on the neural-network output is examined. It is shown to have significant benefits in certain cases when compared with the standard criteria of minimum weighted variance over the branches. In three case studies the ANN-DT algorithm compared favorably with CART, a standard decision tree algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The DSFPN, a new neural network for optical character recognition

    Publication Year: 1999 , Page(s): 1465 - 1473
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (200 KB)  

    A new type of neural network for recognition tasks is presented. The network, called the dynamic supervised forward-propagation network (DSFPN), is based on the forward only version of the counterpropagation network (CPN). The DSFPN, trains using a supervised algorithm and can grow dynamically during training, allowing subclasses in the training data to be learnt in an unsupervised manner. It is shown to train in times comparable to the CPN while giving better classification accuracies than the popular backpropagation network. Both Fourier descriptors and wavelet descriptors are used for image preprocessing and the wavelets are proven to give a far better performance View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimate of exponential convergence rate and exponential stability for neural networks

    Publication Year: 1999 , Page(s): 1487 - 1493
    Cited by:  Papers (58)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    Estimates of exponential convergence rate and exponential stability are studied for a class of neural networks which includes Hopfield neural networks and cellular neural networks. Both local and global exponential convergence are discussed. Theorems for estimation of the exponential convergence rate are established and the bounds on the rate of convergence are given. The domains of attraction in the case of local exponential convergence are obtained. Simple conditions are presented for checking exponential stability of the neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Evolving neural networks to play checkers without relying on expert knowledge

    Publication Year: 1999 , Page(s): 1382 - 1391
    Cited by:  Papers (63)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    An experiment was conducted where neural networks compete for survival in an evolving population based on their ability to play checkers. More specifically, multilayer feedforward neural networks were used to evaluate alternative board positions and games were played using a minimax search strategy. At each generation, the extant neural networks were paired in competitions and selection was used to eliminate those that performed poorly relative to other networks. Offspring neural networks were created from the survivors using random variation of all weights and bias terms. After a series of 250 generations, the best-evolved neural network was played against human opponents in a series of 90 games on an Internet website. The neural network was able to defeat two expert-level players and played to a draw against a master. The final rating of the neural network placed it in the “Class A” category using a standard rating system. Of particular importance in the design of the experiment was the fact that no features beyond the piece differential were given to the neural networks as a priori knowledge. The process of evolution was able to extract all of the additional information required to play at this level of competency. It accomplished this based almost solely on the feedback offered in the final aggregated outcome of each game played (i.e., win, lose, or draw). This procedure stands in marked contrast to the typical artifice of explicitly injecting expert knowledge into a game-playing program View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neural-network method for the nonlinear servomechanism problem

    Publication Year: 1999 , Page(s): 1412 - 1423
    Cited by:  Papers (15)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (292 KB)  

    The solution of the nonlinear servomechanism problem relies on the solvability of a set of mixed nonlinear partial differential and algebraic equations known as the regulator equations. Due to the nonlinear nature, it is difficult to obtain the exact solution of the regulator equations. This paper proposes to solve the regulator equations based on a class of recurrent neural network, which has the features of a cellular neural network. This research not only represents a novel application of the neural networks to numerical mathematics, but also leads to an effective approach to approximately solving the nonlinear servomechanism problem. The resulting design method is illustrated by application to the well-known ball and beam system View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stable dynamic backpropagation learning in recurrent neural networks

    Publication Year: 1999 , Page(s): 1321 - 1334
    Cited by:  Papers (52)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (604 KB)  

    To avoid unstable phenomenon during the learning process, two new learning schemes, called the multiplier and constrained learning rate algorithms, are proposed in this paper to provide stable adaptive updating processes for both the synaptic and somatic parameters of the network. Based on the explicit stability conditions, in the multiplier method these conditions are introduced into the iterative error index, and the new updating formulations contain a set of inequality constraints. In the constrained learning rate algorithm, the learning rate is updated at each iterative instant by an equation derived using the stability conditions. With these stable dynamic backpropagation algorithms, any analog target pattern may be implemented by a steady output vector which is a nonlinear vector function of the stable equilibrium point. The applicability of the approaches presented is illustrated through both analog and binary pattern storage examples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A dynamic channel assignment policy through Q-learning

    Publication Year: 1999 , Page(s): 1443 - 1455
    Cited by:  Papers (24)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    One of the fundamental issues in the operation of a mobile communication system is the assignment of channels to cells and to calls. This paper presents a novel approach to solving the dynamic channel assignment (DCA) problem by using a form of real-time reinforcement learning known as Q-learning in conjunction with neural network representation. Instead of relying on a known teacher the system is designed to learn an optimal channel assignment policy by directly interacting with the mobile communication environment. The performance of the Q-learning based DCA was examined by extensive simulation studies on a 49-cell mobile communication system under various conditions. Comparative studies with the fixed channel assignment (FCA) scheme and one of the best dynamic channel assignment strategies, MAXAVAIL, have revealed that the proposed approach is able to perform better than the FCA in various situations and capable of achieving a performance similar to that achieved by the MAXAVAIL, but with a significantly reduced computational complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Inverting feedforward neural networks using linear and nonlinear programming

    Publication Year: 1999 , Page(s): 1271 - 1290
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (524 KB)  

    The problem of inverting trained feedforward neural networks is to find the inputs which yield a given output. In general, this problem is an ill-posed problem. We present a method for dealing with the inverse problem by using mathematical programming techniques. The principal idea behind the method is to formulate the inverse problem as a nonlinear programming problem, a separable programming (SP) problem, or a linear programming problem according to the architectures of networks to be inverted or the types of network inversions to be computed. An important advantage of the method over the existing iterative inversion algorithm is that various designated network inversions of multilayer perceptrons and radial basis function neural networks can be obtained by solving the corresponding SP problems, which can be solved by a modified simplex method. We present several examples to demonstrate the proposed method and applications of network inversions to examine and improve the generalization performance of trained networks. The results show the effectiveness of the proposed method View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope