By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 2004

Filter Results

Displaying Results 1 - 25 of 27
  • Table of contents

    Page(s): c1 - c4
    Save to Project icon | Request Permissions | PDF file iconPDF (40 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Neural Networks publication information

    Page(s): c2
    Save to Project icon | Request Permissions | PDF file iconPDF (41 KB)  
    Freely Available from IEEE
  • A general framework for learning rules from data

    Page(s): 1333 - 1349
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1045 KB) |  | HTML iconHTML  

    With the aim of getting understandable symbolic rules to explain a given phenomenon, we split the task of learning these rules from sensory data in two phases: a multilayer perceptron maps features into propositional variables and a set of subsequent layers operated by a PAC-like algorithm learns Boolean expressions on these variables. The special features of this procedure are that: i) the neural network is trained to produce a Boolean output having the principal task of discriminating between classes of inputs; ii) the symbolic part is directed to compute rules within a family that is not known a priori; iii) the welding point between the two learning systems is represented by a feedback based on a suitability evaluation of the computed rules. The procedure we propose is based on a computational learning paradigm set up recently in some papers in the fields of theoretical computer science, artificial intelligence and cognitive systems. The present article focuses on information management aspects of the procedure. We deal with the lack of prior information about the rules through learning strategies that affect both the meaning of the variables and the description length of the rules into which they combine. The paper uses the task of learning to formally discriminate among several emotional states as both a working example and a test bench for a comparison with previous symbolic and subsymbolic methods in the field. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Feedforward sigmoidal networks - equicontinuity and fault-tolerance properties

    Page(s): 1350 - 1366
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (461 KB)  

    Sigmoidal feedforward artificial neural networks (FFANNs) have been established to be universal approximators of continuous functions. The universal approximation results are summarized to identify the function sets represented by the sigmoidal FFANNs with the universal approximation properties. The equicontinuous properties of the identified sets is analyzed. The equicontinuous property is related to the fault tolerance of the sigmoidal FFANNs. The generally used arbitrary weight sigmoidal FFANNs are shown to be nonequicontinuous sets. A class of bounded weight sigmoidal FFANNs is established to be equicontinuous. The fault-tolerance behavior of the networks is analyzed and error bounds for the induced errors established. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Context-dependent neural nets-structures and learning

    Page(s): 1367 - 1377
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (648 KB) |  | HTML iconHTML  

    A novel approach toward neural networks modeling is presented in the paper. It is unique in the fact that allows nets' weights to change according to changes of some environmental factors even after completing the learning process. The models of context-dependent (cd) neuron, one- and multilayer feedforward net are presented, with basic learning algorithms and examples of functioning. The Vapnik-Chervonenkis (VC) dimension of a cd neuron is derived, as well as VC dimension of multilayer feedforward nets. Cd nets' properties are discussed and compared with the properties of traditional nets. Possibilities of applications to classification and control problems are also outlined and an example presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Novel direct and self-regulating approaches to determine optimum growing multi-experts network structure

    Page(s): 1378 - 1395
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (883 KB) |  | HTML iconHTML  

    This work presents two novel approaches to determine optimum growing multi-experts network (GMN) structure. The first method called direct method deals with expertise domain and levels in connection with local experts. The growing neural gas (GNG) algorithm is used to cluster the local experts. The concept of error distribution is used to apportion error among the local experts. After reaching the specified size of the network, redundant experts removal algorithm is invoked to prune the size of the network based on the ranking of the experts. However, GMN is not ergonomic due to too many network control parameters. Therefore, a self-regulating GMN (SGMN) algorithm is proposed. SGMN adopts self-adaptive learning rates for gradient-descent learning rules. In addition, SGMN adopts a more rigorous clustering method called fully self-organized simplified adaptive resonance theory in a modified form. Experimental results show SGMN obtains comparative or even better performance than GMN in four benchmark examples, with reduced sensitivity to learning parameters setting. Moreover, both GMN and SGMN outperform the other neural networks and statistical models. The efficacy of SGMN is further justified in three industrial applications and a control problem. It provides consistent results besides holding out a profound potential and promise for building a novel type of nonlinear model consisting of several local linear models. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Contextual processing of structured data by recursive cascade correlation

    Page(s): 1396 - 1410
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1117 KB) |  | HTML iconHTML  

    This paper propose a first approach to deal with contextual information in structured domains by recursive neural networks. The proposed model, i.e., contextual recursive cascade correlation (CRCC), a generalization of the recursive cascade correlation (RCC) model, is able to partially remove the causality assumption by exploiting contextual information stored in frozen units. We formally characterize the properties of CRCC showing that it is able to compute contextual transductions and also some causal supersource transductions that RCC cannot compute. Experimental results on controlled sequences and on a real-world task involving chemical structures confirm the computational limitations of RCC, while assessing the efficiency and efficacy of CRCC in dealing both with pure causal and contextual prediction tasks. Moreover, results obtained for the real-world task show the superiority of the proposed approach versus RCC when exploring a task for which it is not known whether the structural causality assumption holds. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Magnified gradient function with deterministic weight modification in adaptive learning

    Page(s): 1411 - 1423
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (569 KB)  

    This work presents two novel approaches, backpropagation (BP) with magnified gradient function (MGFPROP) and deterministic weight modification (DWM), to speed up the convergence rate and improve the global convergence capability of the standard BP learning algorithm. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function, while the main objective of DWM is to reduce the system error by changing the weights of a multilayered feedforward neural network in a deterministic way. Simulation results show that the performance of the above two approaches is better than BP and other modified BP algorithms for a number of learning problems. Moreover, the integration of the above two approaches forming a new algorithm called MDPROP, can further improve the performance of MGFPROP and DWM. From our simulation results, the MDPROP algorithm always outperforms BP and other modified BP algorithms in terms of convergence rate and global convergence capability. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Hidden space support vector machines

    Page(s): 1424 - 1434
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (681 KB) |  | HTML iconHTML  

    Hidden space support vector machines (HSSVMs) are presented in this paper. The input patterns are mapped into a high-dimensional hidden space by a set of hidden nonlinear functions and then the structural risk is introduced into the hidden space to construct HSSVMs. Moreover, the conditions for the nonlinear kernel function in HSSVMs are more relaxed, and even differentiability is not required. Compared with support vector machines (SVMs), HSSVMs can adopt more kinds of kernel functions because the positive definite property of the kernel function is not a necessary condition. The performance of HSSVMs for pattern recognition and regression estimation is also analyzed. Experiments on artificial and real-world domains confirm the feasibility and the validity of our algorithms. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Encoding nondeterministic fuzzy tree automata into recursive neural networks

    Page(s): 1435 - 1449
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (942 KB) |  | HTML iconHTML  

    Fuzzy neural systems have been a subject of great interest in the last few years, due to their abilities to facilitate the exchange of information between symbolic and subsymbolic domains. However, the models in the literature are not able to deal with structured organization of information, that is typically required by symbolic processing. In many application domains, the patterns are not only structured, but a fuzziness degree is attached to each subsymbolic pattern primitive. The purpose of this paper is to show how recursive neural networks, properly conceived for dealing with structured information, can represent nondeterministic fuzzy frontier-to-root tree automata. Whereas available prior knowledge expressed in terms of fuzzy state transition rules are injected into a recursive network, unknown rules are supposed to be filled in by data-driven learning. We also prove the stability of the encoding algorithm, extending previous results on the injection of fuzzy finite-state dynamics in high-order recurrent networks. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Reproducing chaos by variable structure recurrent neural networks

    Page(s): 1450 - 1457
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (602 KB) |  | HTML iconHTML  

    In this paper, we present a new approach for chaos reproduction using variable structure recurrent neural networks (VSRNN). A neural network identifier is designed, with a variable structure that will change according to its output performance as compared to the given orbits of an unknown chaotic systems. A tradeoff between identification errors and computational complexity is discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Hopfield network learning method for bipartite subgraph problem

    Page(s): 1458 - 1465
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (440 KB)  

    We present a gradient ascent learning method of the Hopfield neural network for bipartite subgraph problem. The method is intended to provide a near-optimum parallel algorithm for solving the bipartite subgraph problem. To do this we use the Hopfield neural network to get a near-maximum bipartite subgraph, and increase the energy by modifying weights in a gradient ascent direction of the energy to help the network escape from the state of the near-maximum bipartite subgraph to the state of the maximum bipartite subgraph or better one. A large number of instances are simulated to verify the proposed method with the simulation results showing that the solution quality is superior to that of best existing parallel algorithm. We also test the learning method on total coloring problem. The simulation results show that our method finds optimal solution in every test graph. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Heterogeneous fuzzy logic networks: fundamentals and development studies

    Page(s): 1466 - 1481
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2045 KB) |  | HTML iconHTML  

    The recent trend in the development of neurofuzzy systems has profoundly emphasized the importance of synergy between the fundamentals of fuzzy sets and neural networks. The resulting frameworks of the neurofuzzy systems took advantage of an array of learning mechanisms primarily originating within the theory of neurocomputing and the use of fuzzy models (predominantly rule-based systems) being well established in the realm of fuzzy sets. Ideally, one can anticipate that neurofuzzy systems should fully exploit the linkages between these two technologies while strongly preserving their evident identities (plasticity or learning abilities to be shared by the transparency and full interpretability of the resulting neurofuzzy constructs). Interestingly, this synergy still becomes a target yet to be satisfied. This study is an attempt to address the fundamental interpretability challenge of neurofuzzy systems. Our underlying conjecture is that the transparency of any neurofuzzy system links directly with the logic fabric of the system so the logic fundamentals of the underlying architecture become of primordial relevance. Having this in mind the development of neurofuzzy models hinges on a collection of logic driven processing units named here fuzzy (logic) neurons. These are conceptually simple logic-oriented elements that come with a well-defined semantics and plasticity. Owing to their diversity, such neurons form essential building blocks of the networks. The study revisits the existing categories of logic neurons, provides with their taxonomy, helps understand their functional features and sheds light on their behavior when being treated as computational components of any neurofuzzy architecture. The two main categories of aggregative and reference neurons are deeply rooted in the fundamental operations encountered in the technology of fuzzy sets (including logic operations, linguistic modifiers, and logic reference operations). The developed heterogeneous networks - - come with a well-defined semantics and high interpretability (which directly translates into the rule-based representation of the networks). As the network takes advantage of various logic neurons, this imposes an immediate requirement of structural optimization, which in this study is addressed by utilizing various mechanisms of genetic optimization (genetic algorithms). We discuss the development of the networks, elaborate on the interpretation aspects and include a number of illustrative numeric examples. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust redesign of a neural network controller in the presence of unmodeled dynamics

    Page(s): 1482 - 1490
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (419 KB) |  | HTML iconHTML  

    This work presents a neural network control redesign, which achieves robust stabilization in the presence of unmodeled dynamics restricted to be input to output practically stable (IOpS), without requiring any prior knowledge on any bounding function. Moreover, the state of the unmodeled dynamics is permitted to go unbounded provided that the nominal system state and/or the control input also go unbounded. The neural network controller is equipped with a resetting strategy to deal with the problem of possible division by zero, which may appear since we consider unknown input vector fields with unknown signs. The uniform ultimate boundedness of the system output to an arbitrarily small set, plus the boundedness of all other signals in the closed-loop is guaranteed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive hybrid control for linear piezoelectric ceramic motor drive using diagonal recurrent CMAC network

    Page(s): 1491 - 1506
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (913 KB) |  | HTML iconHTML  

    This work presents an adaptive hybrid control system using a diagonal recurrent cerebellar-model-articulation-computer (DRCMAC) network to control a linear piezoelectric ceramic motor (LPCM) driven by a two-inductance two-capacitance (LLCC) resonant inverter. Since the dynamic characteristics and motor parameters of the LPCM are highly nonlinear and time varying, an adaptive hybrid control system is therefore designed based on a hypothetical dynamic model to achieve high-precision position control. The architecture of DRCMAC network is a modified model of a cerebellar-model-articulation-computer (CMAC) network to attain a small number of receptive-fields. The novel idea of this study is that it employs the concept of diagonal recurrent neural network (DRNN) in order to capture the system dynamics and convert the static CMAC into a dynamic one. This adaptive hybrid control system is composed of two parts. One is a DRCMAC network controller that is used to mimic a conventional computed torque control law due to unknown system dynamics, and the other is a compensated controller with bound estimation algorithm that is utilized to recover the residual approximation error for guaranteeing the stable characteristic. The effectiveness of the proposed driving circuit and control system is verified with hardware experiments under the occurrence of uncertainties. In addition, the advantages of the proposed control scheme are indicated in comparison with a traditional integral-proportional (IP) position control system. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An adaptive H controller design for bank-to-turn missiles using ridge Gaussian neural networks

    Page(s): 1507 - 1516
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (470 KB) |  | HTML iconHTML  

    A new autopilot design for bank-to-turn (BTT) missiles is presented. In the design of autopilot, a ridge Gaussian neural network with local learning capability and fewer tuning parameters than Gaussian neural networks is proposed to model the controlled nonlinear systems. We prove that the proposed ridge Gaussian neural network, which can be a universal approximator, equals the expansions of rotated and scaled Gaussian functions. Although ridge Gaussian neural networks can approximate the nonlinear and complex systems accurately, the small approximation errors may affect the tracking performance significantly. Therefore, by employing the H control theory, it is easy to attenuate the effects of the approximation errors of the ridge Gaussian neural networks to a prescribed level. Computer simulation results confirm the effectiveness of the proposed ridge Gaussian neural networks-based autopilot with H stabilization. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The pre-image problem in kernel methods

    Page(s): 1517 - 1525
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4413 KB) |  | HTML iconHTML  

    In this paper, we address the problem of finding the pre-image of a feature vector in the feature space induced by a kernel. This is of central importance in some kernel applications, such as on using kernel principal component analysis (PCA) for image denoising. Unlike the traditional method in which relies on nonlinear optimization, our proposed method directly finds the location of the pre-image based on distance constraints in the feature space. It is noniterative, involves only linear algebra and does not suffer from numerical instability or local minimum problems. Evaluations on performing kernel PCA and kernel clustering on the USPS data set show much improved performance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive stochastic resonance in noisy neurons based on mutual information

    Page(s): 1526 - 1540
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (881 KB) |  | HTML iconHTML  

    Noise can improve how memoryless neurons process signals and maximize their throughput information. Such favorable use of noise is the so-called "stochastic resonance" or SR effect at the level of threshold neurons and continuous neurons. This work presents theoretical and simulation evidence that 1) lone noisy threshold and continuous neurons exhibit the SR effect in terms of the mutual information between random input and output sequences, 2) a new statistically robust learning law can find this entropy-optimal noise level, and 3) the adaptive SR effect is robust against highly impulsive noise with infinite variance. Histograms estimate the relevant probability density functions at each learning iteration. A theorem shows that almost all noise probability density functions produce some SR effect in threshold neurons even if the noise is impulsive and has infinite variance. The optimal noise level in threshold neurons also behaves nonlinearly as the input signal amplitude increases. Simulations further show that the SR effect persists for several sigmoidal neurons and for Gaussian radial-basis-function neurons. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A neural network learning for adaptively extracting cross-correlation features between two high-dimensional data streams

    Page(s): 1541 - 1554
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (406 KB) |  | HTML iconHTML  

    This paper proposes a novel cross-correlation neural network (CNN) model for finding the principal singular subspace of a cross-correlation matrix between two high-dimensional data streams. We introduce a novel nonquadratic criterion (NQC) for searching the optimum weights of two linear neural networks (LNN). The NQC exhibits a single global minimum attained if and only if the weight matrices of the left and right neural networks span the left and right principal singular subspace of a cross-correlation matrix, respectively. The other stationary points of the NQC are (unstable) saddle points. We develop an adaptive algorithm based on the NQC for tracking the principal singular subspace of a cross-correlation matrix between two high-dimensional vector sequences. The NQC algorithm provides a fast online learning of the optimum weights for two LNN. The global asymptotic stability of the NQC algorithm is analyzed. The NQC algorithm has several key advantages such as faster convergence, which is illustrated through simulations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fusing images with different focuses using support vector machines

    Page(s): 1555 - 1561
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1092 KB) |  | HTML iconHTML  

    Many vision-related processing tasks, such as edge detection, image segmentation and stereo matching, can be performed more easily when all objects in the scene are in good focus. However, in practice, this may not be always feasible as optical lenses, especially those with long focal lengths, only have a limited depth of field. One common approach to recover an everywhere-in-focus image is to use wavelet-based image fusion. First, several source images with different focuses of the same scene are taken and processed with the discrete wavelet transform (DWT). Among these wavelet decompositions, the wavelet coefficient with the largest magnitude is selected at each pixel location. Finally, the fused image can be recovered by performing the inverse DWT. In this paper, we improve this fusion procedure by applying the discrete wavelet frame transform (DWFT) and the support vector machines (SVM). Unlike DWT, DWFT yields a translation-invariant signal representation. Using features extracted from the DWFT coefficients, a SVM is trained to select the source image that has the best focus at each pixel location, and the corresponding DWFT coefficients are then incorporated into the composite wavelet representation. Experimental results show that the proposed method outperforms the traditional approach both visually and quantitatively. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • New dynamical optimal learning for linear multilayer FNN

    Page(s): 1562 - 1570
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (349 KB)  

    This letter presents a new dynamical optimal learning (DOL) algorithm for three-layer linear neural networks and investigates its generalization ability. The optimal learning rates can be fully determined during the training process. The mean squared error (mse) is guaranteed to be stably decreased and the learning is less sensitive to initial parameter settings. The simulation results illustrate that the proposed DOL algorithm gives better generalization performance and faster convergence as compared to standard error back propagation algorithm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A columnar competitive model for solving combinatorial optimization problems

    Page(s): 1568 - 1574
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (198 KB)  

    The major drawbacks of the Hopfield network when it is applied to some combinatorial problems, e.g., the traveling salesman problem (TSP), are invalidity of the obtained solutions, trial-and-error setting value process of the network parameters and low-computation efficiency. This letter presents a columnar competitive model (CCM) which incorporates winner-takes-all (WTA) learning rule for solving the TSP. Theoretical analysis for the convergence of the CCM shows that the competitive computational neural network guarantees the convergence to valid states and avoids the onerous procedures of determining the penalty parameters. In addition, its intrinsic competitive learning mechanism enables a fast and effective evolving of the network. The simulation results illustrate that the competitive model offers more and better valid solutions as compared to the original Hopfield network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • International Joint Conference on Neural Networks (IJCNN 2005)

    Page(s): 1574
    Save to Project icon | Request Permissions | PDF file iconPDF (138 KB)  
    Freely Available from IEEE
  • The 2nd International Neural Conference on Engineering

    Page(s): 1575
    Save to Project icon | Request Permissions | PDF file iconPDF (413 KB)  
    Freely Available from IEEE
  • Have you visited lately? www.ieee.org [advertisement]

    Page(s): 1576
    Save to Project icon | Request Permissions | PDF file iconPDF (221 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope