By Topic

Neural Networks, IEEE Transactions on

Issue 5 • Date Sept. 1996

Filter Results

Displaying Results 1 - 25 of 27
  • Guest Editorial How to Submit Letters

    Save to Project icon | Request Permissions | PDF file iconPDF (123 KB)  
    Freely Available from IEEE
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent Systems [Book review]

    Save to Project icon | Request Permissions | PDF file iconPDF (105 KB)  
    Freely Available from IEEE
  • Artificial Neural Networks for Modelling and Control of Nonlinear Systems [Book Reviews]

    Save to Project icon | Request Permissions | PDF file iconPDF (105 KB)  
    Freely Available from IEEE
  • A modified HME architecture for text-dependent speaker identification

    Page(s): 1309 - 1313
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (500 KB)  

    A modified hierarchical mixtures of experts (HME) architecture is presented for text-dependent speaker identification. A new gating network is introduced to the original HME architecture for the use of instantaneous and transitional spectral information in text-dependent speaker identification. The statistical model underlying the proposed architecture is presented and learning is treated as a maximum likelihood problem; in particular, an expectation-maximization (EM) algorithm is also proposed for adjusting the parameters of the proposed architecture. An evaluation has been carried out using a database of isolated digit utterances by 10 male speakers. Experimental results demonstrate that the proposed architecture outperforms the original HME architecture in text-dependent speaker identification View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A multilayered perceptron approach to prediction of the SEC's investigation targets

    Page(s): 1286 - 1290
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (516 KB)  

    In the fields of accounting and auditing, detection of firms engaged in fraudulent financial reporting has become increasingly important, due to the increased frequency of such events and the attendant costs of litigation. The neural-network approach sheds some light on this problem due to the attributes that it requires minimum prior knowledge of the data and achieves a highly nonlinear computational model based on past experience (training). In this study, we employ seven red flags which are composed of four financial red flags and three turnover red flags in order to detect targets of the Securities and Exchange Commission's (SECs) investigation of fraudulent financial reporting. The red flags are computed over 70 firms spread among various industrial sectors, and form the base data that is used for developing the computational prediction model. Multilayered perceptron computation of this data was able to predict the targets of the SEC investigated firms with an average of 88% accuracy in the cross-validation test. On the other hand, the same data computed by the logit program gave an average prediction rate of 47% View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Topographic map formation by maximizing unconditional entropy: a plausible strategy for “online” unsupervised competitive learning and nonparametric density estimation

    Page(s): 1299 - 1305
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (768 KB)  

    An unsupervised competitive learning rule, called the vectorial boundary adaptation rule (VBAR), is introduced for topographic map formation. Since VBAR is aimed at producing an equiprobable quantization of the input space, it yields a nonparametric model of the input probability density function. Furthermore, since equiprobable quantization is equivalent to unconditional entropy maximization, we argue that this is a plausible strategy for maximizing mutual information (Shannon information rate) in the case of “online” learning. We use mutual information as a tool for comparing the performance of our rule with Kohonen's self-organizing (feature) map algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Use of bias term in projection pursuit learning improves approximation and convergence properties

    Page(s): 1168 - 1183
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1216 KB)  

    In a regression problem, one is given a multidimensional random vector X, the components of which are called predictor variables, and a random variable, Y, called response. A regression surface describes a general relationship between X and Y. A nonparametric regression technique that has been successfully applied to high-dimensional data is projection pursuit regression (PPR). The regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors. Projection pursuit learning (PPL) formulates PPR using a 2-layer feedforward neural network. The smoothers in PPR are nonparametric, whereas those in PPL are based on Hermite functions of some predefined highest order R. We demonstrate that PPL networks in the original form do not have the universal approximation property for any finite R, and thus cannot converge to the desired function even with an arbitrarily large number of hidden units. But, by including a bias term in each linear projection of the predictor variables, PPL networks can regain these capabilities, independent of the exact choice of R. Experimentally, it is shown in this paper that this modification increases the rate of convergence with respect to the number of hidden units, improves the generalization performance, and makes it less sensitive to the setting of R. Finally, we apply PPL to chaotic time series prediction, and obtain superior results compared with the cascade-correlation architecture View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An optimal tracking neuro-controller for nonlinear dynamic systems

    Page(s): 1099 - 1110
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (912 KB)  

    Multilayer neural networks are used to design an optimal tracking neuro-controller (OTNC) for discrete-time nonlinear dynamic systems with quadratic cost function. The OTNC is made of two controllers: feedforward neuro-controller (FFNC) and feedback neuro-controller (FBNC). The FFNC controls the steady-state output of the plant, while the FBNC controls the transient-state output of the plant. The FFNC is designed using a novel inverse mapping concept by using a neuro-identifier. A generalized backpropagation-through-time (GBTT) algorithm is developed to minimize the general quadratic cost function for the FBNC training. The proposed methodology is useful as an off-line control method where the plant is first identified and then a controller is designed for it. A case study for a typical plant with nonlinear dynamics shows good performance of the proposed OTNC View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multidimensional density shaping by sigmoids

    Page(s): 1291 - 1298
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (652 KB)  

    An estimate of the probability density function of a random vector is obtained by maximizing the output entropy of a feedforward network of sigmoidal units with respect to the input weights. Classification problems can be solved by selecting the class associated with the maximal estimated density. Newton's optimization method, applied to the estimated density, yields a recursive estimator for a random variable or a random sequence. A constrained connectivity structure yields a linear estimator, which is particularly suitable for “real time” prediction. A Gaussian nonlinearity yields a closed-form solution for the network's parameters, which may also be used for initializing the optimization algorithm when other nonlinearities are employed. A triangular connectivity between the neurons and the input, which is naturally suggested by the statistical setting, reduces the number of parameters. Applications to classification and forecasting problems are demonstrated View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Dynamic structure neural networks for stable adaptive control of nonlinear systems

    Page(s): 1151 - 1167
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1468 KB)  

    An adaptive control technique, using dynamic structure Gaussian radial basis function neural networks, that grow in time according to the location of the system's state in space is presented for the affine class of nonlinear systems having unknown or partially known dynamics. The method results in a network that is “economic” in terms of network size, for cases where the state spans only a small subset of state space, by utilizing less basis functions than would have been the case if basis functions were centered on discrete locations covering the whole, relevant region of state space. Additionally, the system is augmented with sliding control so as to ensure global stability if and when the state moves outside the region of state space spanned by the basis functions, and to ensure robustness to disturbances that arise due to the network inherent approximation errors and to the fact that for limiting the network size, a minimal number of basis functions are actually being used. Adaptation laws and sliding control gains that ensure system stability in a Lyapunov sense are presented, together with techniques for determining which basis functions are to form part of the network structure. The effectiveness of the method is demonstrated by experiment simulations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Alien attractors and memory annihilation of structured sets in Hopfield networks

    Page(s): 1305 - 1309
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (528 KB)  

    This paper considers the encoding of structured sets into Hopfield associative memories. A structured set is a set of vectors with equal Hamming distance h from one another, and its centroid is an external vector that has distance h/2 from every vector of the set. Structured sets having centroids are not infrequent. When such a set is encoded into a noiseless Hopfield associative memory using a bipolar outer-product connection matrix, and the network operates with synchronous neuronal update, the memory of all encoded vectors is annihilated even for sets with as few as three vectors in dimension n>5 (four for n=5). In such self-annihilating structured sets, the centroid emerges as a stable attractor. We call it an alien attractor. For canonical structured sets, self-annihilation takes place only if h<n/2. Self-annihilation does not occur and alien attractors do not emerge in dimensions less than five View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Automated learning for reducing the configuration of a feedforward neural network

    Page(s): 1072 - 1085
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1364 KB)  

    In this paper, we present two learning mechanisms for artificial neural networks (ANN's) that can be applied to solve classification problems with binary outputs. These mechanisms are used to reduce the number of hidden units of an ANN when trained by the cascade-correlation learning algorithm (CAS). Since CAS adds hidden units incrementally as learning proceeds, it is difficult to predict the number of hidden units required when convergence is reached. Further, learning must be restarted when the number of hidden units is larger than expected. Our key idea in this paper is to provide alternatives in the learning process and to select the best alternative dynamically based on run-time information obtained. Mixed-mode learning (MM), our first algorithm, provides alternative output matrices so that learning is extended to find one of the many one-to-many mappings instead of finding a unique one-to-one mapping. Since the objective of learning is relaxed by this transformation, the number of learning epochs can be reduced. This in turn leads to a smaller number of hidden units required for convergence. Population-based learning for ANN's (PLAN), our second algorithm, maintains alternative network configurations to select at run time promising networks to train based on error information obtained and time remaining. This dynamic scheduling avoids training possibly unpromising ANNs to completion before exploring new ones. We show the performance of these two mechanisms by applying them to solve the two-spiral problem, a two-region classification problem, and the Pima Indian diabetes diagnosis problem View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Repairs to GLVQ: a new family of competitive learning schemes

    Page(s): 1062 - 1071
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (860 KB)  

    First, we identify an algorithmic defect of the generalized learning vector quantization (GLVQ) scheme that causes it to behave erratically for a certain scaling of the input data. We show that GLVQ can behave incorrectly because its learning rates are reciprocally dependent on the sum of squares of distances from an input vector to the node weight vectors. Finally, we propose a new family of models-the GLVQ-F family-that remedies the problem. We derive competitive learning algorithms for each member of the GLVQ-F model and prove that they are invariant to all scalings of the data. We show that GLVQ-F offers a wide range of learning models since it reduces to LVQ as its weighting exponent (a parameter of the algorithm) approaches one from above. As this parameter increases, GLVQ-F then transitions to a model in which either all nodes may be excited according to their (inverse) distances from an input or in which the winner is excited while losers are penalized. And as this parameter increases without limit, GLVQ-F updates all nodes equally. We illustrate the failure of GLVQ and success of GLVQ-F with the IRIS data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An improved radial basis function network for visual autonomous road following

    Page(s): 1111 - 1120
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1640 KB)  

    We have developed a radial basis function network (RBFN) for visual autonomous road following. Preliminary testing of the RBFN was done using a driving simulator, and the RBFN was then installed on an actual vehicle at Carnegie Mellon University for testing in an outdoor road-following application. In our first attempts, the RBFN had some success, but it experienced some significant problems such as jittery control and driving failure. Several improvements have been made to the original RBFN architecture to overcome these problems in simulation and more importantly in actual road following, and the improvements are described in this paper View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The min-max function differentiation and training of fuzzy neural networks

    Page(s): 1139 - 1150
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1024 KB)  

    This paper discusses the Δ-rule and training of min-max neural networks by developing a differentiation theory for min-max functions, the functions containing min (∧) and/or max (V) operations. We first prove that under certain conditions all min-max functions are continuously differentiable almost everywhere in the real number field ℜ and derive the explicit formulas for the differentiation. These results are the basis for developing the Δ-rule for the training of min-max neural networks. The convergence of the new Δ-rule is proved theoretically using the stochastic theory, and is demonstrated with a simulation example View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Scale-based clustering using the radial basis function network

    Page(s): 1250 - 1261
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1076 KB)  

    This paper shows how scale-based clustering can be done using the radial basis function network (RBFN), with the RBF width as the scale parameter and a dummy target as the desired output. The technique suggests the “right” scale at which the given data set should be clustered, thereby providing a solution to the problem of determining the number of RBF units and the widths required to get a good network solution. The network compares favorably with other standard techniques on benchmark clustering examples. Properties that are required of non-Gaussian basis functions, if they are to serve in alternative clustering networks, are identified. This work, on the whole, points out an important role played by the width parameter in RBFN, when observed over several scales, and provides a fundamental link to the scale space theory developed in computational vision View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Color image processing in a cellular neural-network environment

    Page(s): 1086 - 1098
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2216 KB)  

    When low-level hardware simulations of cellular neural networks (CNNs) are very costly for exploring new applications, the use of a behavioral simulator becomes indispensable. This paper presents a software prototype capable of performing image processing applications using CNNs. The software is based on a CNN multilayer structure in which each primary color is assigned to a unique layer. This allows an added flexibility as different processing applications can be performed in parallel. To be able to handle a full range of color tones, two novel color mapping schemes were derived. In the proposed schemes the color information is obtained from the cell's state rather than from its output. This modification is necessary because for many templates CNN has only binary stable outputs from which only either a fully saturated or a black color can be obtained. Additionally, a postprocessor capable of performing pixelwise logical operations among color layers was developed to enhance the results obtained from CNN. Examples in the areas of medical image processing, image restoration, and weather forecasting are provided to demonstrate the robustness of the software and the vast potential of CNN View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A rapid supervised learning neural network for function interpolation and approximation

    Page(s): 1220 - 1230
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (788 KB)  

    This paper presents a neural-network architecture and an instant learning algorithm that rapidly decides the weights of the designed single-hidden layer neural network. For an n-dimensional N-pattern training set, with a constant bias, a maximum of N-r-1 hidden nodes is required to learn the mapping within a given precision (where r is the rank, usually the dimension, of the input patterns). For off-line training, the proposed network and algorithm is able to achieve “one-shot” training as opposed to most iterative training algorithms in the literature. An online training algorithm is also presented. Similar to most of the backpropagation type of learning algorithms, the given algorithm also interpolates the training data. To eliminate outlier data which may appear in some erroneous training data, a robust weighted least squares method is proposed. The robust weighted least squares learning algorithm can eliminate outlier samples and the algorithm approximates the training data rather than interpolates them. The advantage of the designed network architecture is also mathematically proved. Several experiments show very promising results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Human expression recognition from motion using a radial basis function network architecture

    Page(s): 1121 - 1138
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2452 KB)  

    In this paper a radial basis function network architecture is developed that learns the correlation of facial feature motion patterns and human expressions. We describe a hierarchical approach which at the highest level identifies expressions, at the mid level determines motion of facial features, and at the low level recovers motion directions. Individual expression networks were trained to recognize the “smile” and “surprise” expressions. Each expression network was trained by viewing a set of sequences of one expression for many subjects. The trained neural network was then tested for retention, extrapolation, and rejection ability. Success rates were 88% for retention, 88% for extrapolation, and 83% for rejection View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fuzzy algorithms for learning vector quantization

    Page(s): 1196 - 1211
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1584 KB)  

    This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A four-quadrant subthreshold mode multiplier for analog neural-network applications

    Page(s): 1212 - 1219
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (552 KB)  

    A new four-quadrant CMOS analog multiplier is presented, based on devices operating in the subthreshold mode of conduction. The proposed circuit is a cross-coupled quad structure in which differential multiplication is obtained by driving the gate and bulk (back gate) terminals of the devices. Analysis and simulation have shown that the new structure has the characteristics required for the design of very large scale integration (VLSI) analog neural networks. Although operating at subthreshold current levels, reasonable speed can be obtained since voltage swings are in the range of a few Vt. The behavior of the basic multiplier has been assessed experimentally using transistor-arrays and simulation studies on a network including 11 neurons and 31 synapses indicate a useful level of functionality View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Input-output HMMs for sequence processing

    Page(s): 1231 - 1249
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1956 KB)  

    We consider problems of sequence processing and propose a solution based on a discrete-state model in order to represent past context. We introduce a recurrent connectionist architecture having a modular structure that associates a subnetwork to each state. The model has a statistical interpretation we call input-output hidden Markov model (IOHMM). It can be trained by the estimation-maximization (EM) or generalized EM (GEM) algorithms, considering state trajectories as missing data, which decouples temporal credit assignment and actual parameter estimation. The model presents similarities to hidden Markov models (HMMs), but allows us to map input sequences to output sequences, using the same processing style as recurrent neural networks. IOHMMs are trained using a more discriminant learning paradigm than HMMs, while potentially taking advantage of the EM algorithm. We demonstrate that IOHMMs are well suited for solving grammatical inference problems on a benchmark problem. Experimental results are presented for the seven Tomita grammars, showing that these adaptive models can attain excellent generalization View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Supervised self-coding in multilayered feedforward networks

    Page(s): 1184 - 1195
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1284 KB)  

    Supervised neural-network learning algorithms have proven very successful at solving a variety of learning problems. However, they suffer from a common problem of requiring explicit output labels. This requirement makes such algorithms implausible as biological models. In this paper, it is shown that pattern classification can be achieved, in a multilayered feedforward neural network, without requiring explicit output labels, by a process of supervised self-coding. The class projection is achieved by optimizing appropriate within-class uniformity, and between-class discernability criteria. The mapping function and the class labels are developed together, iteratively using the derived self-coding backpropagation algorithm. The ability of the self-coding network to generalize on unseen data is also experimentally evaluated on real data sets, and compares favorably with the traditional labeled supervision with neural networks. However, interesting features emerge out of the proposed self-coding supervision, which are absent in conventional approaches. The further implications of supervised self-coding with neural networks are also discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope