By Topic

Neural Networks, IEEE Transactions on

Issue 1 • Date Jan. 2002

Filter Results

Displaying Results 1 - 25 of 27
  • In memoriam: Walter J. Karplus

    Page(s): 1
    Save to Project icon | Request Permissions | PDF file iconPDF (9 KB)  
    Freely Available from IEEE
  • Editorial - year 2002: submission options streamlined

    Page(s): 2
    Save to Project icon | Request Permissions | PDF file iconPDF (9 KB)  
    Freely Available from IEEE
  • Comments on "Robust stability for interval Hopfield neural networks with time delay" by X.F. Liao

    Page(s): 250 - 251
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (77 KB) |  | HTML iconHTML  

    The paper points out an unjustified inequality in the paper by X.F. Liao and J.Yu (see ibid., vol.9, p.1042-5, 1998). It concludes that Liao's theorem based on this wrong inequality is not true. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Call for papers

    Page(s): 254
    Save to Project icon | Request Permissions | PDF file iconPDF (17 KB)  
    Freely Available from IEEE
  • NeuroPipe-Chip: A digital neuro-processor for spiking neural networks

    Page(s): 205 - 213
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (259 KB) |  | HTML iconHTML  

    Computing complex spiking artificial neural networks (SANNs) on conventional hardware platforms is far from reaching real-time requirements. Therefore we propose a neuro-processor, called NeuroPipe-Chip, as part of an accelerator board. In this paper, we introduce two new concepts on chip-level to speed up the computation of SANNs. These concepts are implemented in a prototype of the NeuroPipe-Chip. We present the hardware structure of the prototype and evaluate its performance in a system simulation based on a hardware description language (HDL). For the computation of a simple SANN for image segmentation, the NeuroPipe-Chip operating at 100 MHz shows an improvement of more than two orders of magnitude compared to an Alpha 500 MHz workstation and approaches real-time requirements for the computation of SANNs in the order of 106 neurons. Hence, such an accelerator would allow for applications of complex SANNs to solve real-world tasks like real-time image processing. The NeuroPipe-Chip has been fabricated in an Alcatel 0.35-μm digital CMOS technology View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Context in temporal sequence processing: a self-organizing approach and its application to robotics

    Page(s): 45 - 57
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (288 KB) |  | HTML iconHTML  

    A self-organizing neural net for learning and recall of complex temporal sequences is developed and applied to robot trajectory planning. We consider trajectories with both repeated and shared states. Both cases give rise to ambiguities during reproduction of stored trajectories which are resolved via temporal context information. Feedforward weights encode spatial features of the input trajectories, while the temporal order is learned by lateral weights through delayed Hebbian learning. After training, the net model operates in an anticipative fashion by always recalling the successor of the current input state. Redundancy in sequence representation improves noise and fault robustness. The net uses memory resources efficiently by reusing neurons that have previously stored repeated/shared states. Simulations have been carried out to evaluate the performance of the network in terms of trajectory reproduction, convergence time and memory usage, tolerance to fault and noise, and sensitivity to trajectory sampling rate. The results show that the model is fast, accurate, and robust. Its performance is discussed in comparison with other neural-networks models View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Interpretation of artificial neural networks by means of fuzzy rules

    Page(s): 101 - 116
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (394 KB) |  | HTML iconHTML  

    This paper presents an extension of the method presented by Benitez et al (1997) for extracting fuzzy rules from an artificial neural network (ANN) that express exactly its behavior. The extraction process provides an interpretation of the ANN in terms of fuzzy rules. The fuzzy rules presented are in accordance with the domain of the input variables. These rules use a new operator in the antecedent. The properties and intuitive meaning of this operator are studied. Next, the role of the biases in the fuzzy rule-based systems is analyzed. Several examples are presented to comment on the obtained fuzzy rule-based systems. Finally, the interpretation of ANNs with two or more hidden layers is also studied View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural networks with multidimensional transfer functions

    Page(s): 222 - 228
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (181 KB) |  | HTML iconHTML  

    We present a new type of neural network (NN) where the data for the input layer are the value xεR, the vector yε Rm associated to an initial value problem (IVP) with y'(x)= f (y(x)) and a steplength h. Then the stages of a Runge-Kutta (RK) method with trainable coefficients are used as hidden layers for the integration of the IVP using f as transfer function. We take as output two estimations y*, yˆ* of IVP at the point x+h. Training the RK method at some test problems and counting the cost of the method under the coefficients used, we may achieve coefficients that help the method to perform better at a wider class of problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • ViSOM - a novel method for multivariate data projection and structure visualization

    Page(s): 237 - 243
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (129 KB) |  | HTML iconHTML  

    When used for visualization of high-dimensional data, the self-organizing map (SOM) requires a coloring scheme, such as the U-matrix, to mark the distances between neurons. Even so, the structures of the data clusters may not be apparent and their shapes are often distorted. In this paper, a visualization-induced SOM (ViSOM) is proposed to overcome these shortcomings. The algorithm constrains and regularizes the inter-neuron distance with a parameter that controls the resolution of the map. The mapping preserves the inter-point distances of the input data on the map as well as the topology. It produces a graded mesh in the data space such that the distances between mapped data points on the map resemble those in the original space, like in the Sammon mapping. However, unlike the Sammon mapping, the ViSOM can accommodate both training data and new arrivals and is much simpler in computational complexity. Several experimental results and comparisons with other methods are presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Using function approximation to analyze the sensitivity of MLP with antisymmetric squashing activation function

    Page(s): 34 - 44
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB) |  | HTML iconHTML  

    Sensitivity analysis on a neural network is mainly investigated after the network has been designed and trained. Very few have considered this as a critical issue prior to network design. Piche's statistical method (1992, 1995) is useful for multilayer perceptron (MLP) design, but too severe limitations are imposed on both input and weight perturbations. This paper attempts to generalize Piche's method by deriving an universal expression of MLP sensitivity for antisymmetric squashing activation functions, without any restriction on input and output perturbations. Experimental results which are based on, a three-layer MLP with 30 nodes per layer agree closely with our theoretical investigations. The effects of the network design parameters such as the number of layers, the number of neurons per layer, and the chosen activation function are analyzed, and they provide useful information for network design decision-making. Based on the sensitivity analysis of MLP, we present a network design method for a given application to determine the network structure and estimate the permitted weight range for network training View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A two-layer paradigm capable of forming arbitrary decision regions in input space

    Page(s): 15 - 21
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (145 KB) |  | HTML iconHTML  

    It is well known that a two-layer perceptron network with threshold neurons is incapable of forming arbitrary decision regions in input space, while a three-layer perceptron has that capability. The effect of replacing the output neuron in a two-layer perceptron with a bithreshold element is studied. The limitations of this modified two-layer perceptron are observed. Results on the separating capabilities of a pair of parallel hyperplanes are obtained. Based on these, a new two-layer neural paradigm based on increasing the dimensionality of the output of the first layer is proposed and is shown to be capable of forming any arbitrary decision region in input space. Then a type of logic called bithreshold logic, based on the bithreshold neuron transfer function, is studied. Results on the limits of switching function realizability using bithreshold gates are obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Input feature selection for classification problems

    Page(s): 143 - 159
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (411 KB) |  | HTML iconHTML  

    Feature selection plays an important role in classifying systems such as neural networks (NNs). We use a set of attributes which are relevant, irrelevant or redundant and from the viewpoint of managing a dataset which can be huge, reducing the number of attributes by selecting only the relevant ones is desirable. In doing so, higher performances with lower computational effort is expected. In this paper, we propose two feature selection algorithms. The limitation of mutual information feature selector (MIFS) is analyzed and a method to overcome this limitation is studied. One of the proposed algorithms makes more considered use of mutual information between input attributes and output classes than the MIFS. What is demonstrated is that the proposed method can provide the performance of the ideal greedy selection algorithm when information is distributed uniformly. The computational load for this algorithm is nearly the same as that of MIFS. In addition, another feature selection algorithm using the Taguchi method is proposed. This is advanced as a solution to the question as to how to identify good features with as few experiments as possible. The proposed algorithms are applied to several classification problems and compared with MIFS. These two algorithms can be combined to complement each other's limitations. The combined algorithm performed well in several experiments and should prove to be a useful method in selecting features for classification problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Subspace information criterion for nonquadratic regularizers-Model selection for sparse regressors

    Page(s): 70 - 80
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (291 KB) |  | HTML iconHTML  

    Nonquadratic regularizers, in particular the l1 norm regularizer can yield sparse solutions that generalize well. In this work we propose the generalized subspace information criterion (GSIC) that allows to predict the generalization error for this useful family of regularizers. We show that under some technical assumptions GSIC is an asymptotically unbiased estimator of the generalization error. GSIC is demonstrated to have a good performance in experiments with the l1 norm regularizer as we compare with the network information criterion (NIC) and cross- validation in relatively large sample cases. However in the small sample case, GSIC tends to fail to capture the optimal model due to its large variance. Therefore, also a biased version of GSIC is introduced,which achieves reliable model selection in the relevant and challenging scenario of high-dimensional data and few samples View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Maximum likelihood neural approximation in presence of additive colored noise

    Page(s): 117 - 131
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (346 KB) |  | HTML iconHTML  

    In many practical situations, the noise samples may be correlated. In this case, the estimation of noise parameters can be used to improve the approximation. Estimation of the noise structure can also be used to find a stopping criterion in constructive neural networks. To avoid overfitting, a network construction procedure must be stopped when residual can be considered as noise. The knowledge on the noise may be used for "whitening" the residual so that a correlation hypothesis test determines if the network growing must be continued or not. In this paper, supposing a Gaussian noise model, we study the problem of multi-output nonlinear regression using MLP when the noise in each output is a correlated autoregressive time series and is spatially correlated with other output noises. We show that the noise parameters can be determined simultaneously with the network weights and used to construct an estimator with a smaller variance, and so to improve the network generalization performance. Moreover, if a constructive procedure is used to build the network, the estimated parameters may be used to stop the procedure View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An optimal power-dispatching system using neural networks for the electrochemical process of zinc depending on varying prices of electricity

    Page(s): 229 - 236
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (139 KB) |  | HTML iconHTML  

    Depending on varying prices of electricity, an optimal power-dispatching system (OPDS) is developed to minimize the cost of power consumption in the electrochemical process of zinc (EPZ). Due to the complexity of the EPZ, the main factors influencing the power consumption are determined by qualitative analysis, and a series of conditional experiments is conducted to acquire sufficient data, then two backpropagation neural networks are used to describe these relationships quantitatively. An equivalent Hopfield neural network is constructed to solve the optimization problem where a penalty function is introduced into the network energy function so as to meet the equality constraints, and inequality constraints are removed by alteration of the Sigmoid function. This OPDS was put into service in a smeltery in 1998. The cost of power consumption has decreased significantly, the total electrical energy consumption is reduced, and it is also beneficial to balancing the load of the power grid. The actual results show the effectiveness of the OPDS. This paper introduces a successful industrial application and mainly presents how to utilize neural networks to solve particular problems for the real world View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Effect of transmission delay on the rate of convergence of a class of nonlinear contractive dynamical systems

    Page(s): 244 - 248
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (223 KB) |  | HTML iconHTML  

    We investigate the qualitative properties of a general class of contractive dynamical systems with time delay by using a unified analysis approach for any p-contraction with p ∈ [1,∞]. It is proved that the delayed contractive dynamical system is always globally exponentially stable no matter how large the time delay is, while the rate of convergence of the delayed system is reduced as the time delay increases. A lower bound on the rate of convergence of the delayed contractive dynamical system is obtained, which is the unique positive solution of a nonlinear equation with three parameters, namely, the time delay, the time constant and the p-contraction constant in the system. We show that the previously established results in the literature about the global asymptotic or exponential stability independent of delay for Hopfield-type neural networks can actually be deduced by recasting the network model into the general framework of contractive dynamical systems with some p-contraction (p ∈ [1,∞]) under the given delay-independent stability conditions. Numerical simulation examples are also presented to illustrate the obtained theoretical results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A general backpropagation algorithm for feedforward neural networks learning

    Page(s): 251 - 254
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (100 KB) |  | HTML iconHTML  

    A general backpropagation algorithm is proposed for feedforward neural network learning with time varying inputs. The Lyapunov function approach is used to rigorously analyze the convergence of weights, with the use of the algorithm, toward minima of the error function. Sufficient conditions to guarantee the convergence of weights for time varying inputs are derived. It is shown that most commonly used backpropagation learning algorithms are special cases of the developed general algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • μARTMAP: use of mutual information for category reduction in Fuzzy ARTMAP

    Page(s): 58 - 69
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (356 KB) |  | HTML iconHTML  

    A new architecture called μARTMAP is proposed to impact a category proliferation problem present in Fuzzy ARTMAP. Under a probabilistic setting, it seeks a partition of the input space that optimizes the mutual information with the output space, but allowing some training error, thus avoiding overfitting. It implements an inter-ART reset mechanism that permits handling exceptions correctly, thus using few categories, especially in high dimensionality problems. It compares favorably to Fuzzy ARTMAP and Boosted ARTMAP in several synthetic benchmarks, being more robust to noise than Fuzzy ARTMAP and degrading less as dimensionality increases. Evaluated on a real-world task, the recognition of handwritten characters, it performs comparably to Fuzzy ARTMAP, while generating a much more compact rule set View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neuron-adaptive higher order neural-network models for automated financial data modeling

    Page(s): 188 - 204
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (419 KB) |  | HTML iconHTML  

    Real-world financial data is often nonlinear, comprises high-frequency multipolynomial components, and is discontinuous (piecewise continuous). Not surprisingly, it is hard to model such data. Classical neural networks are unable to automatically determine the optimum model and appropriate order for financial data approximation. We address this problem by developing neuron-adaptive higher order neural-network (NAHONN) models. After introducing one-dimensional (1-D), two-dimensional (2-D), and n-dimensional NAHONN models, we present an appropriate learning algorithm. Network convergence and the universal approximation capability of NAHONNs are also established. NAHONN Group models (NAHONGs) are also introduced. Both NAHONNs and NAHONGs are shown to be "open box" and as such are more acceptable to financial experts than classical (closed box) neural networks. These models are further shown to be capable of automatically finding not only the optimum model, but also the appropriate order for specific financial data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Direct adaptive NN control of a class of nonlinear systems

    Page(s): 214 - 221
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (216 KB) |  | HTML iconHTML  

    In this paper, direct adaptive neural-network (NN) control is presented for a class of affine nonlinear systems in the strict-feedback form with unknown nonlinearities. By utilizing a special property of the affine term, the developed scheme,avoids the controller singularity problem completely. All the signals in the closed loop are guaranteed to be semiglobally uniformly ultimately bounded and the output of the system is proven to converge to a small neighborhood of the desired trajectory. The control performance of the closed-loop system is guaranteed by suitably choosing the design parameters. Simulation results are presented to show the effectiveness of the approach View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive multilayer perceptrons with long- and short-term memories

    Page(s): 22 - 33
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (245 KB) |  | HTML iconHTML  

    Multilayer perceptrons (MLPs) with long- and short-term memories (LASTMs) are proposed for adaptive processing. The activation functions of the output neurons of such a network are linear, and thus the weights in the last layer affect the outputs of the network linearly and are called linear weights. These linear weights constitute the short-term memory and other weights the long-term memory. It is proven that virtually any function f(x, θ) with an environmental parameter θ can be approximated to any accuracy by an MLP with LASTMs whose long-term memory is independent of θ. This independency of θ allows the long-term memory to be determined in an a priori training and allows the online adjustment of only the short-term memory for adapting to the environmental parameter θ. The benefits of using an MLP with LASTMs include less online computation, no poor local extrema to fall into, and much more timely and better adaptation. Numerical examples illustrate that these benefits are realized satisfactorily View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Energy function-based approaches to graph coloring

    Page(s): 81 - 91
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (310 KB) |  | HTML iconHTML  

    We describe an approach to optimization based on a multiple-restart quasi-Hopfield network where the only problem-specific knowledge is embedded in the energy function that the algorithm tries to minimize. We apply this method to three different variants of the graph coloring problem: the minimum coloring problem, the spanning subgraph k-coloring problem, and the induced subgraph k-coloring problem. Though Hopfield networks have been applied in the past to the minimum coloring problem, our encoding is more natural and compact than almost all previous ones. In particular, we use k-state neurons while almost all previous approaches use binary neurons. This reduces the number of connections in the network from (Nk)2 to N2 asymptotically and also circumvents a problem in earlier approaches, that of multiple colors being assigned to a single vertex. Experimental results show that our approach compares favorably with other algorithms, even nonneural ones specifically developed for the graph coloring problem View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new clustering technique for function approximation

    Page(s): 132 - 142
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (248 KB) |  | HTML iconHTML  

    To date, clustering techniques have always been oriented to solve classification and pattern recognition problems. However, some authors have applied them unchanged to construct initial models for function approximators. Nevertheless, classification and function approximation problems present quite different objectives. Therefore it is necessary to design new clustering algorithms specialized in the problem of function approximation. This paper presents a new clustering technique, specially designed for function. approximation problems, which improves the performance of the approximator system obtained, compared with other models derived from traditional classification oriented clustering algorithms and input-output clustering techniques View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Basic dynamics from a pulse-coupled network of autonomous integrate-and-fire chaotic circuits

    Page(s): 92 - 100
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (330 KB)  

    This paper studies basic dynamics from a novel pulse-coupled network (PCN). The unit element of the PCN is an integrate-and-fire circuit (IFC) that exhibits chaos. We an give an iff condition for the chaos generation. Using two IFC, we construct a master-slave PCN. It exhibits interesting chaos synchronous phenomena and their breakdown phenomena. We give basic classification of the phenomena and their existence regions can be elucidated in the parameter space. We then construct a ring-type PCN and elucidate that the PCN exhibits interesting grouping phenomena based on the chaos synchronization patterns. Using a simple test circuit, some of typical phenomena can be verified in the laboratory View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The MCA EXIN neuron for the minor component analysis

    Page(s): 160 - 187
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (675 KB) |  | HTML iconHTML  

    The minor component analysis (MCA) deals with the recovery of the eigenvector associated to the smallest eigenvalue of the autocorrelation matrix of the input data and is a very important tool for signal processing and data analysis. It is almost exclusively solved by linear neurons. This paper presents a linear neuron endowed with a novel learning law, called MCA EXINn and analyzes its features. The neural literature about MCA is very poor, in the sense that both a little theoretical basis is given (almost always focusing on the ODE asymptotic approximation) and only experiments on toy problems (at most four-dimensional problems) are presented, without any numerical analysis. This work addresses these problems and lays sound theoretical foundations for the neural MCA theory. In particular, it classifies the MCA neurons according to the Riemannian metric and justifies, from the analysis of the degeneracy of the error cost; the different behavior in approaching convergence. The cost landscape is studied and used as a basis for the analysis of the asymptotic behavior. All the phases of the dynamics of the MCA algorithms are investigated in detail and, together with the numerical analysis, lead to the identification of three possible kinds of divergence, here called sudden, dynamic, and numerical. The importance of the choice of low initial conditions is also explained. A lot of importance is given to the experimental part, where simulations on high-dimensional problems are,presented and analyzed. The orthogonal regression or total least squares (TLS) technique is also presented, together with a real-world application on the identification of the parameters of an electrical machine. It can be concluded that MCA EXIN is the best MCA neuron in terms of stability (no finite time divergence), speed, and accuracy View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope