By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 1998

Filter Results

Displaying Results 1 - 25 of 47
  • Comments on "Functional equivalence between radial basis function networks and fuzzy inference systems" [with reply]

    Page(s): 1529 - 1532
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (80 KB)  

    In the original paper of Jang and Sun (ibid., vol.4, p. 156-9, 1993), it is claimed that under a set of minor restrictions radial basis function networks and fuzzy inference systems are functionally equivalent. The paper shows that this set of restrictions is incomplete and that, when it is completed, the said functional equivalence applies only to a small range of fuzzy inference systems. In addition, a modified set of restrictions is proposed which is applicable for a much wider range of fuzzy inference systems. The original authors reply that they concerned themselves only with input-output functions, and not with the whole of learning. They also draw attention to the intention of their paper, written at a time when the field of study was much less mature. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Author's reply

    Page(s): 1531 - 1532
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (28 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Call for papers

    Page(s): 1532
    Save to Project icon | Request Permissions | PDF file iconPDF (5 KB)  
    Freely Available from IEEE
  • List of reviewers

    Page(s): 1535 - 1536
    Save to Project icon | Request Permissions | PDF file iconPDF (8 KB)  
    Freely Available from IEEE
  • Author index

    Page(s): 1 - 5
    Save to Project icon | Request Permissions | PDF file iconPDF (201 KB)  
    Freely Available from IEEE
  • Subject index

    Page(s): 5
    Save to Project icon | Request Permissions | PDF file iconPDF (117 KB)  
    Freely Available from IEEE
  • Index

    Page(s): 5 - 16
    Save to Project icon | Request Permissions | PDF file iconPDF (230 KB)  
    Freely Available from IEEE
  • Nonlinear blind equalization schemes using complex-valued multilayer feedforward neural networks

    Page(s): 1442 - 1455
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (584 KB)  

    Among the useful blind equalization algorithms, stochastic-gradient iterative equalization schemes are based on minimizing a nonconvex and nonlinear cost function. However, as they use a linear FIR filter with a convex decision region, their residual estimation error is high. In the paper, four nonlinear blind equalization schemes that employ a complex-valued multilayer perceptron instead of the linear filter are proposed and their learning algorithms are derived. After the important properties that a suitable complex-valued activation function must possess are discussed, a new complex-valued activation function is developed for the proposed schemes to deal with QAM signals of any constellation sizes. It has been further proven that by the nonlinear transformation of the proposed function, the correlation coefficient between the real and imaginary parts of input data decreases when they are jointly Gaussian random variables. Last, the effectiveness of the proposed schemes is verified in terms of initial convergence speed and MSE in the steady state. In particular, even without carrier phase tracking procedure, the proposed schemes correct an arbitrary phase rotation caused by channel distortion View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A tight bound on concept learning

    Page(s): 1191 - 1202
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (480 KB)  

    A tight bound on the generalization performance of concept learning is shown by a novel approach. Unlike existing theories, the new approach uses no assumption on large sample size as in Bayesian approach and does not consider the uniform learnability as in the VC dimension analysis. We analyze the generalization performance of some particular learning algorithm that is not necessarily well behaved, in the hope that once learning curves or sample complexity of this algorithm is obtained, it is applicable to real learning situations. The result is expressed in a dimension called Boolean interpolation dimension, and is tight in the sense that it meets the lower bound requirement of Baum and Haussler (1989). The Boolean interpolation dimension is not greater than the number of modifiable system parameters, and definable for almost all the real-world networks such as backpropagation networks and linear threshold multilayer networks. It is shown that the generalization error follows from a beta distribution of parameters m, the number of training examples, and d, the Boolean interpolation dimension. This implies that for large d, the learning results tend to the average-case result, known as the self-averaging property of the learning. The bound is shown to be applicable to the practical learning algorithms that can be modeled by the Gibbs algorithm with a uniform prior. The result is also extended to the case of inconsistent learning View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new approach to artificial neural networks

    Page(s): 1167 - 1179
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (276 KB)  

    A novel approach to artificial neural networks is presented. The philosophy of this approach is based on two aspects: the design of task-specific networks, and a new neuron model with multiple synapses. The synapses' connective strengths are modified through selective and cumulative processes conducted by axo-axonic connections from a feedforward circuit. This new concept was applied to the position control of a planar two-link manipulator exhibiting excellent results on learning capability and generalization when compared with a conventional feedforward network. In the present paper, the example shows only a network developed from a neuronal reflexive circuit with some useful artifices, nevertheless without the intention of covering all possibilities devised View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the closure of the set of functions that can be realized by a given multilayer perceptron

    Page(s): 1086 - 1098
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (784 KB)  

    Given a multilayer perceptron (MLP) with a fixed architecture, there are functions that can be approximated up to any degree of accuracy, without having to increase the number of the hidden nodes. Those functions belong to the closure F¯ of the set F¯ of the maps realizable by the MLP. In this paper, we give a list of maps with this property. In particular, it is proven that: 1) rational functions belongs to F¯ for networks with inverse tangent activation function; and 2) products of polynomials and exponentials belongs to F¯ for networks with sigmoid activation function. Moreover, for a restricted class of MLPs, we prove that the list is complete and give an analytic definition of F¯ View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A nonlinear discriminant algorithm for feature extraction and data classification

    Page(s): 1370 - 1376
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (244 KB)  

    Presents a nonlinear supervised feature extraction algorithm that combines Fisher's criterion function with a preliminary perceptron-like nonlinear projection of vectors in pattern space. Its main motivation is to combine the approximation properties of multilayer perceptrons (MLPs) with the target free nature of Fisher's classical discriminant analysis. In fact, although MLPs provide good classifiers for many problems, there may be some situations, such as unequal class sizes with a high degree of pattern mixing among them, that may make difficult the construction of good MLP classifiers. In these instances, the features extracted by our procedure could be more effective. After the description of its construction and the analysis of its complexity, we illustrate its use over a synthetic problem with the above characteristics View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • TMLNN: triple-valued or multiple-valued logic neural network

    Page(s): 1099 - 1117
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (688 KB)  

    We discuss the problem of representing and processing triple-valued or multiple-valued logic knowledge using neural network. A novel neuron model, triple-valued or multiple-valued logic neuron (TMLN), is presented. Each TMLN can represent a triple-valued or multiple-valued logic rule by itself. We will show that there are two TMLNs: TMLN-AND (triple-valued or multiple-valued “logic AND”) neuron and TMLN-OR (triple-valued or multiple-valued “logic OR”) neuron. Two simplified TMLN models are also presented, and show that a multilayer neural network made up of triple-valued or multiple-valued logic neurons (TMLNN) can implement a triple-valued or multiple-valued logic inference system. The training algorithm for TMLNN is presented and can be shown to converge. In our model, triple-valued or multiple-valued logic rules can be extracted from TMLNN with ease. TMLNN can thus form a base for representing logic knowledge using neural network View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Searching for optimal frame patterns in an integrated TDMA communication system using mean field annealing

    Page(s): 1292 - 1300
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (268 KB)  

    In an integrated time-division multiple access (TDMA) communication system, voice and data are multiplexed in time to share a common transmission link in a frame format in which time is divided into slots. A certain number of time slots in a frame are allocated to voice and the rest are used to transmit data. Maximum data throughput can be achieved by searching for the optimal configuration(s) of relative positions of voice and data transmissions in a frame (frame pattern). When the problem size becomes large, the computational complexity in searching for the optimal patterns becomes intractable. In the paper, mean field annealing (MFA), which provides near-optimal solutions with reasonable complexity, is proposed to solve this problem. The determination of the related parameters are addressed. Comparison with the random search and simulated annealing algorithm is made in terms of solution optimality and computational complexity. Simulation results show that the MFA approach exhibits a good tradeoff between performance and computational complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • CARVE-a constructive algorithm for real-valued examples

    Page(s): 1180 - 1190
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (336 KB)  

    A constructive neural-network algorithm is presented. For any consistent classification task on real-valued training vectors, the algorithm constructs a feedforward network with a single hidden layer of threshold units which implements the task. The algorithm, which we call CARVE, extends the “sequential learning” algorithm of Marchand et al. (1990) from Boolean inputs to the real-valued input case, and uses convex hull methods for the determination of the network weights. The algorithm is an efficient training scheme for producing near-minimal network solutions for arbitrary classification tasks. The algorithm is applied to a number of benchmark problems including German and Sejnowski's sonar data, the Monks problems and Fisher's iris data. A significant application of the constructive algorithm is in providing an initial network topology and initial weights for other neural-network training schemes, and this is demonstrated by application to backpropagation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Competitive neural trees for pattern classification

    Page(s): 1352 - 1369
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB)  

    Presents competitive neural trees (CNeTs) for pattern classification. The CNeT contains m-ary nodes and grows during learning by using inheritance to initialize new nodes. At the node level, the CNeT employs unsupervised competitive learning. The CNeT performs hierarchical clustering of the feature vectors presented to it as examples, while its growth is controlled by forward pruning. Because of the tree structure, the prototype in the CNeT close to any example can be determined by searching only a fraction of the tree. The paper introduces different search methods for the CNeT, which are utilized for training as well as for recall. The CNeT is evaluated and compared with existing classifiers on a variety of pattern classification problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

    Page(s): 1057 - 1068
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (156 KB)  

    To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN's but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e,g., recurrent neural networks) and explanation structures. In addition we identify some of the key research questions in extracting the knowledge embedded within ANN's including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural network-based control design: an LMI approach

    Page(s): 1422 - 1429
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB)  

    We address a neural network-based control design for a discrete-time nonlinear system. Our design approach is to approximate the nonlinear system with a multilayer perceptron of which the activation functions are of the sigmoid type symmetric to the origin. A linear difference inclusion representation is then established for this class of approximating neural networks and is used to design a state feedback control law for the nonlinear system based on the certainty equivalence principle. The control design equations are shown to be a set of linear matrix inequalities where a convex optimization algorithm can be applied to determine the control signal. Further, the stability of the closed-loop is guaranteed in the sense that there exists a unique global attraction region in the neighborhood of the origin to which every trajectory of the closed-loop system converges. Finally, a simple example is presented so as to illustrate our control design procedure View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Shape-adaptive radial basis functions

    Page(s): 1155 - 1166
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (364 KB)  

    Radial basis functions for discrimination and regression have been used with some success in a wide variety of applications. Here, we investigate the optimal choice for the form of the basis functions and present an iterative strategy for obtaining the function in a regression context using a conjugate gradient-based algorithm together with a nonparametric smoother. This is developed in a discrimination framework using the concept of optimal scaling. Results are presented for a range of simulated and real data sets View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Prediction limit estimation for neural network models

    Page(s): 1515 - 1522
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (256 KB)  

    A novel method for estimation of prediction limits for global and local approximating neural networks is presented. The method partitions the input space using self-organizing feature maps to introduce the concept of local neighborhoods, and calculates limits that indicate the extent to which one can rely on predictions for making future decisions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems

    Page(s): 1388 - 1408
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1136 KB)  

    A feedback control law is proposed that drives the controlled vector vt of a discrete-time dynamic system to track a reference vt* over an infinite time horizon, while minimizing a given cost function. The behavior of vt* over time is completely unpredictable, Random noises act on the dynamic system and the state observation channel, which may be nonlinear. It is assumed that all such random vectors are mutually independent, and that their probability density functions are known. So general a non-LQG optimal control problem is very difficult to solve. The proposed solution is based on three main approximating assumptions: 1) the problem is stated in a receding-horizon framework where vt* is assumed to remain constant within a shifting-time window; 2) the control law is assigned a given structure (that of a multilayer feedforward neural net) in which a finite number of parameters have to be determined so as to minimize the cost function; and 3) the control law is given a limited memory, which prevents the amount of data to be stored from increasing over time. Errors resulting from the second and third assumptions are discussed, Due to the very general assumptions under which the control law is derived, we are not able to report stability results. However, simulation results show that the proposed method may constitute an effective tool for solving, to a sufficient degree of accuracy, a wide class of control problems traditionally regarded as difficult ones. An example of freeway traffic optimal control is given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Comparing neural networks: a benchmark on growing neural gas, growing cell structures, and fuzzy ARTMAP

    Page(s): 1279 - 1291
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (260 KB)  

    Compares the performance of some incremental neural networks with the well-known multilayer perceptron (MLP) on real-world data. The incremental networks are fuzzy ARTMAP (FAM), growing neural gas (GNG) and growing cell structures (GCS). The real-world datasets consist of four different datasets posing different challenges to the networks in terms of complexity of decision boundaries, overlapping between classes, and size of the datasets. The performance of the networks on the datasets is reported with respect to measure classification error, number of training epochs, and sensitivity toward variation of parameters. Statistical evaluations are applied to examine the significance of the results. The overall performance ranks in the following descending order: GNG, GCS, MLP, FAM View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Periodic symmetric functions, serial addition, and multiplication with neural networks

    Page(s): 1118 - 1128
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (392 KB)  

    This paper investigates threshold based neural networks for periodic symmetric Boolean functions and some related operations. It is shown that any n-input variable periodic symmetric Boolean function can be implemented with a feedforward linear threshold-based neural network with size of O(log n) and depth also of O(log n), both measured in terms of neurons. The maximum weight and fan-in values are in the order of O(n). Under the same assumptions on weight and fan-in values, an asymptotic bound of O(log n) for both size and depth of the network is also derived for symmetric Boolean functions that can be decomposed into a constant number of periodic symmetric Boolean subfunctions. Based on this results neural networks for serial binary addition and multiplication of n-bit operands are also proposed. It is shown that the serial addition can be computed with polynomially bounded weights and a maximum fan-in in the order of O(log n) in O(n/log n) serial cycles. Finally, it is shown that the serial multiplication can be computed in O(n) serial cycles with O(log n) size neural gate network, and with O(n log n) latches View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A common neural-network model for unsupervised exploratory data analysis and independent component analysis

    Page(s): 1495 - 1501
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (236 KB)  

    This paper presents the derivation of an unsupervised learning algorithm, which enables the identification and visualization of latent structure within ensembles of high-dimensional data. This provides a linear projection of the data onto a lower dimensional subspace to identify the characteristic structure of the observations independent latent causes. The algorithm is shown to be a very promising tool for unsupervised exploratory data analysis and data visualization. Experimental results confirm the attractiveness of this technique for exploratory data analysis and an empirical comparison is made with the recently proposed generative topographic mapping (GTM) and standard principal component analysis (PCA). Based on standard probability density models a generic nonlinearity is developed which allows both (1) identification and visualization of dichotomised clusters inherent in the observed data and (2) separation of sources with arbitrary distributions from mixtures, whose dimensionality may be greater than that of number of sources. The resulting algorithm is therefore also a generalized neural approach to independent component analysis (ICA) and it is considered to be a promising method for analysis of real-world data that will consist of sub- and super-Gaussian components such as biomedical signals View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope