By Topic

Neural Networks, IEEE Transactions on

Issue 4 • Date July 1995

Filter Results

Displaying Results 1 - 22 of 22
  • Comments on "Learning convergence in the cerebellar model articulation controller"

    Page(s): 1016 - 1018
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (284 KB)  

    The commenter refers to the paper by Wong-Sideris (ibid. vol.3, p.115-21 (1992)) claiming that the original Albus CMAC (or binary CMAC) is capable of learning an arbitrary multivariate lookup table, the linear optimization process is strictly positive definite, and that the basis functions are linearly independent, given sufficient training data. In recent work by Brown et al. (1994), however, it has been proved that the multivariate binary CMAC is unable to learn certain multivariate lookup tables and the number of such orthogonal functions increases exponentially as the generalization parameter is increased. A simple 2D orthogonal function is presented as a counterexample to the original theory. It is also demonstrated that the basis functions are-always linearly dependent, both for the univariate and the multivariate case, and hence the linear optimization process is only positive semi-definite and there always exists an infinite number of possible optimal weight vectors.<> View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems

    Page(s): 911 - 917
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (520 KB)  

    The purpose of this paper is to investigate neural network capability systematically. The main results are: 1) every Tauber-Wiener function is qualified as an activation function in the hidden layer of a three-layered neural network; 2) for a continuous function in S'(R1 ) to be a Tauber-Wiener function, the necessary and sufficient condition is that it is not a polynomial; 3) the capability of approximating nonlinear functionals defined on some compact set of a Banach space and nonlinear operators has been shown; and 4) the possibility by neural computation to approximate the output as a whole (not at a fixed point) of a dynamical system, thus identifying the system View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the computational power of Elman-style recurrent networks

    Page(s): 1000 - 1004
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (404 KB)  

    Recently, Elman (1991) has proposed a simple recurrent network which is able to identify and classify temporal patterns. Despite the fact that Elman networks have been used extensively in many different fields, their theoretical capabilities have not been completely defined. Research in the 1960's showed that for every finite state machine there exists a recurrent artificial neural network which approximates it to an arbitrary degree of precision. This paper extends that result to architectures meeting the constraints of Elman networks, thus proving that their computational power is as great as that of finite state machines View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A competitive associative memory model and its dynamics

    Page(s): 929 - 940
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (904 KB)  

    Conventional associative memory networks perform “noncompetitive recognition” or “competitive recognition in distance”. In this paper a “competitive recognition” associative memory model is introduced which simulates the competitive persistence of biological species. Unlike most of the conventional networks, the proposed model takes only the prototype patterns as its equilibrium points, so that the spurious points are effectively excluded. Furthermore, it is shown that, as the competitive parameters vary, the network has a unique stable equilibrium point corresponding to the winner competitive parameter and, in this case, the unique stable equilibrium state can be recalled from any initial key View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sensitivity analysis of single hidden-layer neural networks with threshold functions

    Page(s): 1005 - 1007
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    An important consideration when applying neural networks to pattern recognition is the sensitivity to weight perturbation or to input errors. In this paper, we analyze the sensitivity of single hidden-layer networks with threshold functions. In a case of weight perturbation or input errors, the probability of inversion error for an output neuron is derived as a function of the trained weights, the input pattern, and the variance of weight perturbation or the bit error probability of the input pattern. The derived results are verified with a simulation of the Madaline recognizing handwritten digits. The result shows that the sensitivity of trained networks is far different from that of networks with random weights View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • ART-EMAP: A neural network architecture for object recognition by evidence accumulation

    Page(s): 805 - 818
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1356 KB)  

    A new neural network architecture is introduced for the recognition of pattern classes after supervised and unsupervised learning. Applications include spatio-temporal image understanding and prediction and 3D object recognition from a series of ambiguous 2D views. The architecture, called ART-EMAP, achieves a synthesis of adaptive resonance theory (ART) and spatial and temporal evidence integration for dynamic predictive mapping (EMAP). ART-EMAP extends the capabilities of fuzzy ARTMAP in four incremental stages. Stage 1 introduces distributed pattern representation at a view category field. Stage 2 adds a decision criterion to the mapping between view and object categories, delaying identification of ambiguous objects when faced with a low confidence prediction. Stage 3 augments the system with a field where evidence accumulates in medium-term memory. Stage 4 adds an unsupervised learning process to fine-tune performance after the limited initial period of supervised network training. Each ART-EMAP stage is illustrated with a benchmark simulation example, using both noisy and noise-free data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Emergent synchrony in locally coupled neural oscillators

    Page(s): 941 - 948
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (736 KB)  

    The discovery of long range synchronous oscillations in the visual cortex has triggered much interest in understanding the underlying neural mechanisms and in exploring possible applications of neural oscillations. Many neural models thus proposed end up relying on global connections, leading to the question of whether lateral connections alone can produce remote synchronization. With a formulation different from frequently used phase models, we find that locally coupled neural oscillators can yield global synchrony. The model employs a previously suggested mechanism that the efficacy of the connections is allowed to change on a fast time scale. Based on the known connectivity of the visual cortex, the model outputs closely resemble the experimental findings. Furthermore, we illustrate the potential of locally connected oscillator networks in perceptual grouping and pattern segmentation, which seems missing in globally connected ones View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neural network implementation using a single MOST per synapse

    Page(s): 1008 - 1011
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (372 KB)  

    A VLSI implementation of an artificial neural network using a single n-channel MOS (metal-oxide semiconductor) transistor per synapse is investigated. The simplicity of the design is achieved by using pulse width modulation to represent neural activity and by using a novel technique to manipulate negative weights. A simple multilayer perceptron (MLP) network was simulated using the SPICE circuit simulator and the performance of a hardware realization of the same MLP network was measured. Simulations and measurements are shown to agree well View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Active control of vibration using a neural network

    Page(s): 819 - 828
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (936 KB)  

    Feedforward control of sound and vibration using a neural network-based control system is considered, with the aim being to derive an architecture/algorithm combination which is capable of supplanting the commonly used finite impulse response filter/filtered-x least mean square (LMS) linear arrangement for certain nonlinear problems. An adaptive algorithm is derived which enables stable adaptation of the neural controller for this purpose, while providing the capacity to maintain causality within the control scheme. The algorithm is shown to be simply a generalization of the linear filtered-x LMS algorithm. Experiments are undertaken which demonstrate the utility of the proposed arrangement, showing that it performs as well as a linear control system for a linear control problem and better for a nonlinear control problem. The experiments also lead to the conclusion that more work is required to improve the predictability and consistency of the performance before the neural network controller becomes a practical alternative to the current linear feedforward systems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Using Taguchi's method of experimental design to control errors in layered perceptrons

    Page(s): 949 - 961
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1140 KB)  

    A significant problem in the design and construction of an artificial neural network for function approximation is limiting the magnitude and the variance of errors when the network is used in the field. Network errors can occur when the training data does not faithfully represent the required function due to noise or low sampling rates, when the network's flexibility does not match the variability of the data, or when the input data to the resultant network is noisy. This paper reports on several experiments whose purpose was to rank the relative significance of these error sources and thereby find neural network design principles for limiting the magnitude and variance of network errors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An analysis of the GLVQ algorithm

    Page(s): 1012 - 1016
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    Generalized learning vector quantization (GLVQ) has been proposed in as a generalization of the simple competitive learning (SCL) algorithm. The main argument of GLVQ proposal is its superior insensitivity to the initial values of the weights (code vectors). In this paper we show that the distinctive characteristics of the definition of GLVQ disappear outside a small domain of applications. GLVQ becomes identical to SCL when either the number of code vectors grows or the size of the input space is large. Besides that, the behavior of GLVQ is inconsistent for problems defined on very small scale input spaces. The adaptation rules fluctuate between performing descent and ascent searches on the gradient of the distortion function View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis and design of an analog sorting network

    Page(s): 962 - 971
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (788 KB)  

    An analog sorting neural network is presented. First, existing order representations are discussed and a generalized order representation is introduced. The sorting problem is then formulated as the assignment problem. Based on the assignment problem formulation, the neural network architecture is described. Design principles and an op-amp based circuit realization of the analog neural network are delineated. Three illustrative examples are also discussed to demonstrate the capability and performance of the analog neural network. The proposed analog neural network is shown to be capable of monotonic and bitonic sorting and suitable for hardware implementation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An evidential extension of the MRII training algorithm for detecting erroneous MADALINE responses

    Page(s): 880 - 892
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1308 KB)  

    This paper integrates the evidential reasoning methodology with the parallel distributed learning paradigm of artificial neural networks (ANN). As such, this work presents an algorithm for the detection and, if possible, subsequent correction of the errors in the neuron responses in the output layer of the multiple adaptive linear element (MADALINE) ANN. A geometrical perspective of the MADALINE ANN processing methodology is provided. This perspective is then used to formulate a statistical specification to identify and quantify the sources of uncertainties in the MADALINE processing methodology. A new algorithm, EMRII, is then developed as an extension to the original MRII (MADELINE rule II) algorithm, to formulate support and plausibility measures based on the statistical specification. The support and plausibility measures, thus formulated, are indicative of the degree of confidence of the ANN, in regards to the correctness of its outputs. Based on the support measure, a scheme utilizing two thresholds is proposed to facilitate the interpretation of the support values for error prediction in the ANN responses. Finally, simulation results for the application of the EMRII algorithm in the prediction of erroneous responses in an example problem is presented. These simulation results highlight the error detection capabilities of the EMRII algorithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution

    Page(s): 829 - 836
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (792 KB)  

    It is often difficult to predict the optimal neural network size for a particular application. Constructive or destructive methods that add or subtract neurons, layers, connections, etc. might offer a solution to this problem. We prove that one method, recurrent cascade correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities. It cannot represent with monotone (i.e., sigmoid) and hard-threshold activation functions certain finite state automata. We give a “preliminary” approach on how to get around these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure. We illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A framework for improved training of Sigma-Pi networks

    Page(s): 893 - 903
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (912 KB)  

    This paper proposes and demonstrates a framework for Sigma-Pi networks such that the combinatorial increase in product terms is avoided. This is achieved by only implementing a subset of the possible product terms (sub-net Sigma-Pi). Application of a dynamic weight pruning algorithm enables redundant weights to be removed and replaced during the learning process, hence permitting access to a larger weight space than employed at network initialization. More than one learning rate is applied to ensure that the inclusion of higher order descriptors does not result in over description of the training set (memorization). The application of such a framework is tested using a problem requiring significant generalization ability. Performance of the resulting sub-net Sigma-Pi network is compared to that returned by optimal multi-layer perceptrons and general Sigma-Pi solutions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning in linear neural networks: a survey

    Page(s): 837 - 858
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2060 KB)  

    Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and self-organization can sometimes be answered analytically. We survey most of the known results on linear networks, including: 1) backpropagation learning and the structure of the error function landscape, 2) the temporal evolution of generalization, and 3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A multistage neural network for color constancy and color induction

    Page(s): 972 - 985
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1364 KB)  

    A biologically-based multistage neural network is presented which produces color constant responses to a variety of color stimuli. The network takes advantage of several mechanisms in the human visual system, including retinal adaptation, spectral opponency, and spectrally-specific long-range inhibition. This last stage is a novel mechanism based on cells which have been described in cortical area V4. All stages include nonlinear response functions. The model emulates human performance in several psychophysical paradigms designed to test color constancy and color induction. We measured the amount of constancy achieved with both natural and artificial simulated illuminants, using homogeneous grey backgrounds and more complex backgrounds, such as Mondrians. On average, the model performs as well or better than the average human color constancy performance under similar conditions. The network simulation also displays color induction and assimilation behavior consistent with human perceptual data View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The decision-making properties of discrete multiple exponential bidirectional associative memories

    Page(s): 993 - 999
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (444 KB)  

    A method for modeling the learning of belief combination in evidential reasoning using a neural network is presented. A centralized network composed of multiple exponential bidirectional associative memories (eBAM's) sharing a single output array of neurons is proposed to process the uncertainty management of many pieces of evidence simultaneously. The stability of the proposed multiple eBAM network is proved. The sufficient condition to recall a stored pattern pair is discussed. Most important of all, a majority rule of decision making in presentation of multiple evidence is also found by the study of signal-noise-ratio of multiple eBAM network. A guaranteed stable state condition, i.e., the condition for the fastest recall of a pattern pair, is also studied. The result is coherent with the intuition of reasoning View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks

    Page(s): 904 - 910
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (544 KB)  

    The purpose of this paper is to explore the representation capability of radial basis function (RBF) neural networks. The main results are: 1) the necessary and sufficient condition for a function of one variable to be qualified as an activation function in RBF network is that the function is not an even polynomial, and 2) the capability of approximation to nonlinear functionals and operators by RBF networks is revealed, using sample data either in frequency domain or in time domain, which can be used in system identification by neural networks View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Neurocontrollers trained with rules extracted by a genetic assisted reinforcement learning system

    Page(s): 859 - 879
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1544 KB)  

    This paper proposes a novel system for rule extraction of temporal control problems and presents a new way of designing neurocontrollers. The system employs a hybrid genetic search and reinforcement learning strategy for extracting the rules. The learning strategy requires no supervision and no reference model. The extracted rules are weighted micro rules that operate on small neighborhoods of the admissable control space. A further refinement of the extracted rules is achieved by applying additional genetic search and reinforcement to reduce the number of extracted micro rules. This process results in a smaller set of macro rules which can be used to train a feedforward multilayer perceptron neurocontroller. The micro rules or the macro rules may also be utilized directly in a table look-up controller. As an example of the macro rules-based neurocontroller, we chose four benchmarks. In the first application we verify the capability of our system to learn optimal linear control strategies. The other three applications involve engine idle speed control, bioreactor control, and stabilizing two poles on a moving cart. These problems are highly nonlinear, unstable, and may include noise and delays in the plant dynamics. In terms of retrievals; the neurocontrollers generally outperform the controllers using a table look-up method. Both controllers, though, show robustness against noise disturbances and plant parameter variations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Observer-participant models of neural processing

    Page(s): 918 - 928
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (804 KB)  

    A model is proposed in which the neuron serves as an information channel. Channel distortion occurs through the channel since the mapping from input Boolean codes to output codes are many-to-one in that neuron outputs consist of just two distinguished states. Within the described model, the neuron performs a decision-making function. Decisions are made regarding the validity of a question passively posed by the neuron. This question becomes defined through learning hence learning is viewed as the process of determining an appropriate question based on supplied input ensembles. An application of the Shannon information measures of entropy and mutual information taken together in the context of the proposed model lead to the Hopfield neuron model with conditionalized Hebbian learning rules. Neural decisions are shown to be based on a sigmoidal transfer characteristic or, in the limit as computational temperature tends to zero, a maximum likelihood decision rule. The described work is contrasted with the information-theoretic approach of Linsker View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A supervised learning neural network coprocessor for soft-decision maximum-likelihood decoding

    Page(s): 986 - 992
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (572 KB)  

    A supervised learning neural network (SLNN) coprocessor which enhances the performance of a digital soft-decision Viterbi decoder used for forward error correction in a digital communication channel with either fading plus additive white Gaussian noise (AWGN) or pure AWGN has been investigated and designed. The SLNN is designed to cooperate with a phase shift keying (PSK) demodulator, an automatic gain control (AGC) circuit, and a 3-bit quantizer which is an analog to digital convertor. It is trained to learn the best uniform quantization step-size Δ BEST as a function of the mean and the standard deviation of various sets of Gaussian distributed random variables. The channel cutoff rate (R0) of the channel is employed to determine the best quantization threshold step-size (ΔBEST) that results in the minimization of the Viterbi decoder output bit error rate (BER). For a digital communication system with a SLNN coprocessor, consistent and substantial BER performance improvements are observed. The performance improvement ranges from a minimum of 9% to a maximum of 25% for a pure AWGN channel and from a minimum of 25% to a maximum of 70% for a fading channel. This neural network coprocessor approach can be generalized and applied to any digital signal processing system to decrease the performance losses associated with quantization and/or signal instability View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope