By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 1999

Filter Results

Displaying Results 1 - 25 of 32
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author index

    Page(s): 1 - 6
    Save to Project icon | Request Permissions | PDF file iconPDF (216 KB)  
    Freely Available from IEEE
  • Subject index

    Page(s): 6 - 20
    Save to Project icon | Request Permissions | PDF file iconPDF (231 KB)  
    Freely Available from IEEE
  • A dynamic channel assignment policy through Q-learning

    Page(s): 1443 - 1455
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    One of the fundamental issues in the operation of a mobile communication system is the assignment of channels to cells and to calls. This paper presents a novel approach to solving the dynamic channel assignment (DCA) problem by using a form of real-time reinforcement learning known as Q-learning in conjunction with neural network representation. Instead of relying on a known teacher the system is designed to learn an optimal channel assignment policy by directly interacting with the mobile communication environment. The performance of the Q-learning based DCA was examined by extensive simulation studies on a 49-cell mobile communication system under various conditions. Comparative studies with the fixed channel assignment (FCA) scheme and one of the best dynamic channel assignment strategies, MAXAVAIL, have revealed that the proposed approach is able to perform better than the FCA in various situations and capable of achieving a performance similar to that achieved by the MAXAVAIL, but with a significantly reduced computational complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonlinear adaptive trajectory tracking using dynamic neural networks

    Page(s): 1402 - 1411
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (292 KB)  

    In this paper the adaptive nonlinear identification and trajectory tracking are discussed via dynamic neural networks. By means of a Lyapunov-like analysis we determine stability conditions for the identification error. Then we analyze the trajectory tracking error by a local optimal controller. An algebraic Riccati equation and a differential one are used for the identification and the tracking error analysis. As our main original contributions, we establish two theorems: the first one gives a bound for the identification error, and the second one establishes a bound for the tracking error. We illustrate the effectiveness of these results by two examples: the second-order relay system with multiple isolated equilibrium points and the chaotic system given by Duffing equation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Training neural networks with additive noise in the desired signal

    Page(s): 1511 - 1517
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (116 KB)  

    A global optimization strategy for training adaptive systems such as neural networks and adaptive filters (finite or infinite impulse response) is proposed. Instead of adding random noise to the weights as proposed in the past, additive random noise is injected directly into the desired signal. Experimental results show that this procedure also speeds up greatly the backpropagation algorithm. The method is very easy to implement in practice, preserving the backpropagation algorithm and requiring a single random generator with a monotonically decreasing step size per output channel. Hence, this is an ideal strategy to speed up supervised learning, and avoid local minima entrapment when the noise variance is appropriately scheduled View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Complex cell prototype representation for face recognition

    Page(s): 1528 - 1531
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    We propose a face recognition system based on a biologically inspired filtering method. Our work differs from previous proposals in: 1) the multistage filtering method employed; 2) the pyramid structure used, and most importantly; 3) the prototype construction scheme to determine the models stored in memory. The method is much simpler than previous proposals and relatively inexpensive computationally, while attaining error rates as low as 5%, very close to the best reported results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • ANN-DT: an algorithm for extraction of decision trees from artificial neural networks

    Page(s): 1392 - 1401
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (176 KB)  

    Although artificial neural networks can represent a variety of complex systems with a high degree of accuracy, these connectionist models are difficult to interpret. This significantly limits the applicability of neural networks in practice, especially where a premium is placed on the comprehensibility or reliability of systems. A novel artificial neural-network decision tree algorithm (ANN-DT) is therefore proposed, which extracts binary decision trees from a trained neural network. The ANN-DT algorithm uses the neural network to generate outputs for samples interpolated from the training data set. In contrast to existing techniques, ANN-DT can extract rules from feedforward neural networks with continuous outputs. These rules are extracted from the neural network without making assumptions about the internal structure of the neural network or the features of the data. A novel attribute selection criterion based on a significance analysis of the variables on the neural-network output is examined. It is shown to have significant benefits in certain cases when compared with the standard criteria of minimum weighted variance over the branches. In three case studies the ANN-DT algorithm compared favorably with CART, a standard decision tree algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimate of exponential convergence rate and exponential stability for neural networks

    Page(s): 1487 - 1493
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    Estimates of exponential convergence rate and exponential stability are studied for a class of neural networks which includes Hopfield neural networks and cellular neural networks. Both local and global exponential convergence are discussed. Theorems for estimation of the exponential convergence rate are established and the bounds on the rate of convergence are given. The domains of attraction in the case of local exponential convergence are obtained. Simple conditions are presented for checking exponential stability of the neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Exploring constructive cascade networks

    Page(s): 1335 - 1350
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (416 KB)  

    Constructive algorithms have proved to be powerful methods for training feedforward neural networks. An important property of these algorithms is generalization. A series of empirical studies were performed to examine the effect of regularization on generalization in constructive cascade algorithms. It was found that the combination of early stopping and regularization resulted in better generalization than the use of early stopping alone. A cubic penalty term that greatly penalizes large weights was shown to be beneficial for generalization in cascade networks. An adaptive method of setting the regularization magnitude in constructive algorithms was introduced and shown to produce generalization results similar to those obtained with a fixed, user-optimized regularization setting. This adaptive method also resulted in the construction of smaller networks for more complex problems. The acasper algorithm, which incorporates the insights obtained from the empirical studies, was shown to have good generalization and network construction properties. This algorithm was compared to the cascade correlation algorithm on the Proben 1 and additional regression data sets View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Design quality and robustness with neural networks

    Page(s): 1518 - 1527
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (180 KB)  

    Data sets that are gathered for industrial applications are frequently noisy and present challenges that necessitate use of different methodologies. We present a successful example of generating concise and accurate neural-network models for multiple quality characteristics in injection molding. These models map process measurements to product quality. They are used for product and process design, addressing material and processing issues, and defining the operating window and its robustness. The models are developed based on data from designed and other experiments. Linear regression, decision tree induction, nonlinear regression, as well as “stepwise neural networks” were used for feature selection and model comparison. The final model consists of a neural network with three inputs, one hidden layer and five outputs, modeling five critical to quality variables simultaneously with high accuracy. The neural network was visualized for validation and insight View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stable dynamic backpropagation learning in recurrent neural networks

    Page(s): 1321 - 1334
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (604 KB)  

    To avoid unstable phenomenon during the learning process, two new learning schemes, called the multiplier and constrained learning rate algorithms, are proposed in this paper to provide stable adaptive updating processes for both the synaptic and somatic parameters of the network. Based on the explicit stability conditions, in the multiplier method these conditions are introduced into the iterative error index, and the new updating formulations contain a set of inequality constraints. In the constrained learning rate algorithm, the learning rate is updated at each iterative instant by an equation derived using the stability conditions. With these stable dynamic backpropagation algorithms, any analog target pattern may be implemented by a steady output vector which is a nonlinear vector function of the stable equilibrium point. The applicability of the approaches presented is illustrated through both analog and binary pattern storage examples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neural-network method for the nonlinear servomechanism problem

    Page(s): 1412 - 1423
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (292 KB)  

    The solution of the nonlinear servomechanism problem relies on the solvability of a set of mixed nonlinear partial differential and algebraic equations known as the regulator equations. Due to the nonlinear nature, it is difficult to obtain the exact solution of the regulator equations. This paper proposes to solve the regulator equations based on a class of recurrent neural network, which has the features of a cellular neural network. This research not only represents a novel application of the neural networks to numerical mathematics, but also leads to an effective approach to approximately solving the nonlinear servomechanism problem. The resulting design method is illustrated by application to the well-known ball and beam system View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Relative loss bounds for single neurons

    Page(s): 1291 - 1304
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (348 KB)  

    We analyze and compare the well-known gradient descent algorithm and the more recent exponentiated gradient algorithm for training a single neuron with an arbitrary transfer function. Both algorithms are easily generalized to larger neural networks, and the generalization of gradient descent is the standard backpropagation algorithm. We prove worst-case loss bounds for both algorithms in the single neuron case. Since local minima make it difficult to prove worst case bounds for gradient-based algorithms, we must use a loss function that prevents the formation of spurious local minima. We define such a matching loss function for any strictly increasing differentiable transfer function and prove worst-case loss bounds for any such transfer function and its corresponding matching loss. The different forms of the two algorithms' bounds indicates that exponentiated gradient outperforms gradient descent when the inputs contain a large number of irrelevant components. Simulations on synthetic data confirm these analytical results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Toward an optimal PRNN-based nonlinear predictor

    Page(s): 1435 - 1442
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (176 KB)  

    We present an approach for selecting optimal parameters for the pipelined recurrent neural network (PRNN) in the paradigm of nonlinear and nonstationary signal prediction. We consider the role of nesting, which is inherent to the PRNN architecture. The corresponding number of nested modules needed for a certain prediction task, and their contribution toward the final prediction gain give a thorough insight into the way the PRNN performs, and offers solutions for optimization of its parameters. In particular, nesting allows the forgetting factor in the cost function of the PRNN to exceed unity, hence it becomes an emphasis factor. This compensates for the small contribution of the distant modules to the prediction process, due to nesting, and helps to circumvent the problem of vanishing gradient, experienced in RNNs for prediction. The PRNN is shown to outperform the linear least mean square and recursive least squares predictors, as well as previously proposed PRNN schemes, at no expense of additional computational complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Incremental learning methods with retrieving of interfered patterns

    Page(s): 1351 - 1365
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (476 KB)  

    There are many cases when a neural-network-based system must memorize some new patterns incrementally. However, if the network learns the new patterns only by referring to them, it probably forgets old memorized patterns, since parameters in the network usually correlate not only to the old memories but also to the new patterns. A certain way to avoid the loss of memories is to learn the new patterns with all memorized patterns. It needs, however, a large computational power. To solve this problem, we propose incremental learning methods with retrieval of interfered patterns (ILRI). In these methods, the system employs a modified version of a resource allocating network (RAN) which is one variation of a generalized radial basis function (GRBF). In ILRI, the RAN learns new patterns with a relearning of a few number of retrieved past patterns that are interfered with the incremental learning. We construct ILRI in two steps. In the first step, we construct a system which searches the interfered patterns from past input patterns stored in a database. In the second step, we improve the first system in such a way that the system does not need the database. In this case, the system regenerates the input patterns approximately in a random manner. The simulation results show that these two systems have almost the same ability, and the generalization ability is higher than other similar systems using neural networks and k-nearest neighbors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • RBF principal manifolds for process monitoring

    Page(s): 1424 - 1434
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    This paper describes a novel means for creating a nonlinear extension of principal component analysis (PCA) using radial basis function (RBF) networks. This algorithm comprises two distinct stages: projection and self-consistency. The projection stage contains a single network, trained to project data from a high- to a low-dimensional space. Training requires solution of a generalized eigenvector equation. The second stage, trained using a novel hybrid nonlinear optimization algorithm, then performs the inverse transformation. Issues relating to the practical implementation of the procedure are discussed, and the algorithm is demonstrated on a nonlinear test problem. An example of the application of the algorithm to data from a benchmark simulation of an industrial overheads condenser and reflux drum rig is also included. This shows the usefulness of the procedure in detecting and isolating both sensor and process faults. Pointers for future research in this area are also given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic behavior of irreducible excitatory networks of analog graded-response neurons

    Page(s): 1375 - 1381
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (220 KB)  

    In irreducible excitatory networks of analog graded-response neurons, the trajectories of most solutions tend to the equilibria. We derive sufficient conditions for such networks to be globally asymptotically stable. When the network possesses several locally stable equilibria, their location in the phase space is discussed and a description of their attraction basin is given. The results hold even when interunit transmission is delayed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Shape recovery from shading by a new neural-based reflectance model

    Page(s): 1536 - 1541
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB)  

    We present a neural-based reflectance model of which the physical parameters of the reflectivity under different lighting conditions are interpreted by the network weights. The idea of our method is to optimize a proper reflectance model by an effective learning algorithm and to recover the object surface by a simple shape from shading recursive algorithm with this resulting model. Experiments, including synthetic and real images, were performed to demonstrate the performance of the proposed method for practical applications View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The “weight smoothing” regularization of MLP for Jacobian stabilization

    Page(s): 1502 - 1510
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (208 KB)  

    In an approximation problem with a neural network, a low-output root mean square error is not always a universal criterion. We investigate problems where the Jacobians-first derivative of an output value with respect to an input value-of the approximation model are needed and propose to add a quality criterion on these Jacobians during the learning step. More specifically, we focus on the approximation of functionals 𝒜, from a space of continuous functions (discretized in practice) to a scalar space. In this case, the approximation is confronted with the compensation phenomenon: a lower contribution of one input can be compensated by a larger one of its neighboring inputs. In this case, profiles (with respect to the input index) of neural Jacobians are very irregular instead of smooth. Then, the approximation of 𝒜 becomes an ill-posed problem because many solutions can be chosen by the learning process. We propose to introduce the smoothness of Jacobian profiles as an a priori information via a regularization technique and develop a new and efficient learning algorithm, called “weight smoothing”. We assess the robustness of the weight smoothing algorithm by testing it on a real and complex problem stemming from meteorology: the neural approximation of the forward model of radiative transfer equation in the atmosphere. The stabilized Jacobians of this model are then used in an inversion process to illustrate the improvement of the Jacobians after weight smoothing View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An adaptable time-delay neural-network algorithm for image sequence analysis

    Page(s): 1531 - 1536
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (564 KB)  

    We present an algorithm based on a time-delay neural network with spatio-temporal receptive fields and adaptable time delays for image sequence analysis. Our main result is that tedious manual adaptation of the temporal size of the receptive fields can be avoided by employing a method to adapt the corresponding time delay and related network structure parameters during the training process View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiple neural-network-based adaptive controller using orthonormal activation function neural networks

    Page(s): 1494 - 1501
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (212 KB)  

    A direct adaptive control scheme is developed using orthonormal activation function-based neural networks (OAFNNs) for trajectory tracking control of a class of nonlinear systems. Multiple OAFNNs are employed in these controllers for feedforward compensation of unknown system dynamics. Choice of multiple OAFNNs allows a reduction in overall network size reducing the computational requirements. The network weights are tuned online, in real time. The overall stability of the system and the neural networks is guaranteed using Lyapunov analysis. The developed neural controllers are evaluated experimentally and the experimental results are shown to support theoretical analysis. The effects of network parameters on system performance are experimentally evaluated and are presented. The superior learning capability of OAFNNs is demonstrated through experimental results. The OAFNNs were able to model the true nature of the nonlinear system dynamics characteristics for a rolling-sliding contact as well as for stiction View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sample selection via clustering to construct support vector-like classifiers

    Page(s): 1474 - 1481
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    Explores the possibility of constructing RBF classifiers which, somewhat like support vector machines, use a reduced number of samples as centroids, by means of selecting samples in a direct way. Because sample selection is viewed as a hard computational problem, this selection is done after a previous vector quantization: this way obtains also other similar machines using centroids selected from those that are learned in a supervised manner. Several forms of designing these machines are considered, in particular with respect to sample selection; as well as some different criteria to train them. Simulation results for well-known classification problems show very good performance of the corresponding designs, improving that of support vector machines and reducing substantially their number of units. This shows that our interest in selecting samples (or centroids) in an efficient manner is justified. Many new research avenues appear from these experiments and discussions, as suggested in our conclusions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bayesian approach to neural-network modeling with input uncertainty

    Page(s): 1261 - 1270
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (208 KB)  

    It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise or corruption. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural-network framework which allows for input noise provided that some model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method gives an additional term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable and sampling this jointly with the network weights using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Partition-based and sharp uniform error bounds

    Page(s): 1315 - 1320
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (128 KB)  

    This paper develops probabilistic bounds on out-of-sample error rates for several classifiers using a single set of in-sample data. The bounds are based on probabilities over partitions of the union of in-sample and out-of-sample data into in-sample and out-of-sample data sets, The bounds apply when in-sample and out-of-sample data are drawn from the same distribution. Partition-based bounds are stronger than the Vapnik-Chervonenkis bounds, but they require more computation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope