By Topic

Neural Networks, IEEE Transactions on

Issue 6 • Date Nov. 2002

Filter Results

Displaying Results 1 - 25 of 30
  • Author index

    Page(s): 1563 - 1567
    Save to Project icon | Request Permissions | PDF file iconPDF (207 KB)  
    Freely Available from IEEE
  • Subject index

    Page(s): 1567 - 1579
    Save to Project icon | Request Permissions | PDF file iconPDF (255 KB)  
    Freely Available from IEEE
  • Fuzzy kernel perceptron

    Page(s): 1364 - 1373
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1480 KB) |  | HTML iconHTML  

    A new learning method, the fuzzy kernel perceptron (FKP), in which the fuzzy perceptron (FP) and the Mercer kernels are incorporated, is proposed in this paper. The proposed method first maps the input data into a high-dimensional feature space using some implicit mapping functions. Then, the FP is adopted to find a linear separating hyperplane in the high-dimensional feature space. Compared with the FP, the FKP is more suitable for solving the linearly nonseparable problems. In addition, it is also more efficient than the kernel perceptron (KP). Experimental results show that the FKP has better classification performance than FP, KP, and the support vector machine. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • All fiber-optic neural network using coupled SOA based ring lasers

    Page(s): 1504 - 1513
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (425 KB) |  | HTML iconHTML  

    An all-optical neural network is presented that is based on coupled lasers. Each laser in the network lases at a distinct wavelength, representing one neuron. The network status is determined by the wavelength of the network's light output. Inputs to the network are in the optical power domain. The nonlinear threshold function required for neural-network operation is achieved optically by interaction between the lasers. The behavior of the coupled lasers is explained by a simple laser model developed in the paper. In particular, the winner take all (WTA) neural-network behavior of a system of many lasers is described. An experimental system is implemented using single mode fiber optic components at wavelengths near 1550 nm. A number of functions are implemented to demonstrate the practicality of the new network. The neural network is particularly robust against input wavelength variations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Hybrid intelligent systems for time series prediction using neural networks, fuzzy logic, and fractal theory

    Page(s): 1395 - 1408
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1662 KB)  

    In this paper, we describe a new method for the estimation of the fractal dimension of a geometrical object using fuzzy logic techniques. The fractal dimension is a mathematical concept, which measures the geometrical complexity of an object. The algorithms for estimating the fractal dimension calculate a numerical value using as data a time series for the specific problem. This numerical (crisp) value gives an idea of the complexity of the geometrical object (or time series). However, there is an underlying uncertainty in the estimation of the fractal dimension because we use only a sample of points of the object, and also because the numerical algorithms for the fractal dimension are not completely accurate. For this reason, we have proposed a new definition of the fractal dimension that incorporates the concept of a fuzzy set. This new definition can be considered a weaker definition (but more realistic) of the fractal dimension, and we have named this the "fuzzy fractal dimension." We can apply this new definition of the fractal dimension in conjunction with soft computing techniques for the problem of time series prediction. We have developed hybrid intelligent systems combining neural networks, fuzzy logic, and the fractal dimension, for the problem of time series prediction, and we have achieved very good results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The growing hierarchical self-organizing map: exploratory analysis of high-dimensional data

    Page(s): 1331 - 1341
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (831 KB)  

    The self-organizing map (SOM) is a very popular unsupervised neural-network model for the analysis of high-dimensional input data as in data mining applications. However, at least two limitations have to be noted, which are related to the static architecture of this model as well as to the limited capabilities for the representation of hierarchical relations of the data. With our novel growing hierarchical SOM (GHSOM) we address both limitations. The GHSOM is an artificial neural-network model with hierarchical architecture composed of independent growing SOMs. The motivation was to provide a model that adapts its architecture during its unsupervised training process according to the particular requirements of the input data. Furthermore, by providing a global orientation of the independently growing maps in the individual layers of the hierarchy, navigation across branches is facilitated. The benefits of this novel neural network are a problem-dependent architecture and the intuitive representation of hierarchical relations in the data. This is especially appealing in explorative data mining applications, allowing the inherent structure of the data to unfold in a highly intuitive fashion. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fully automatic clustering system

    Page(s): 1285 - 1298
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (602 KB) |  | HTML iconHTML  

    In this paper, the fully automatic clustering system (FACS) is presented. It is a technique for clustering and vector quantization whose objective is the automatic calculation of the codebook of the right dimension, the desired error (or target) being fixed. At each iteration, FACS tries to improve the setting of the existing codewords and, if necessary, some elements are removed from or added to the codebook. In order to save on the number of computations per iteration, greedy techniques are adopted. It has been demonstrated, from a heuristic point of view, that the number of the codewords determined by FACS is very low and that the algorithm quickly converges toward the final solution. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Effects of moving the center's in an RBF network

    Page(s): 1299 - 1307
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (894 KB) |  | HTML iconHTML  

    In radial basis function (RBF) networks, placement of centers is said to have a significant effect on the performance of the network. Supervised learning of center locations in some applications show that they are superior to the networks whose centers are located using unsupervised methods. But such networks can take the same training time as that of sigmoid networks. The increased time needed for supervised learning offsets the training time of regular RBF networks. One way to overcome this may be to train the network with a set of centers selected by unsupervised methods and then to fine tune the locations of centers. This can be done by first evaluating whether moving the centers would decrease the error and then, depending on the required level of accuracy, changing the center locations. This paper provides new results on bounds for the gradient and Hessian of the error considered first as a function of the independent set of parameters, namely the centers, widths, and weights; and then as a function of centers and widths where the linear weights are now functions of the basis function parameters for networks of fixed size. Moreover, bounds for the Hessian are also provided along a line beginning at the initial set of parameters. Using these bounds, it is possible to estimate how much one can reduce the error by changing the centers. Further to that, a step size can be specified to achieve a guaranteed, amount of reduction in error. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Face recognition by independent component analysis

    Page(s): 1450 - 1464
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1405 KB)  

    A number of current face recognition algorithms use face representations found by unsupervised statistical methods. Typically these methods find a set of basis images and represent faces as a linear combination of those images. Principal component analysis (PCA) is a popular example of such methods. The basis images found by PCA depend only on pairwise relationships between pixels in the image database. In a task such as face recognition, in which important information may be contained in the high-order relationships among pixels, it seems reasonable to expect that better basis images may be found by methods sensitive to these high-order statistics. Independent component analysis (ICA), a generalization of PCA, is one such method. We used a version of ICA derived from the principle of optimal information transfer through sigmoidal neurons. ICA was performed on face images in the FERET database under two different architectures, one which treated the images as random variables and the pixels as outcomes, and a second which treated the pixels as random variables and the images as outcomes. The first architecture found spatially local basis images for the faces. The second architecture produced a factorial face code. Both ICA representations were superior to representations based on PCA for recognizing faces across days and changes in expression. A classifier that combined the two ICA representations gave the best performance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the discrete-time dynamics of the basic Hebbian neural network node

    Page(s): 1342 - 1352
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1081 KB) |  | HTML iconHTML  

    In this paper, the dynamical behavior of the basic node used for constructing Hebbian artificial neural networks (NNs) is analyzed. Hebbian NNs are employed in communications and signal processing applications, among others. They have been traditionally studied on a continuous-time formulation whose validity is justified via some analytical procedures that presume, among other hypotheses, a specific asymptotic behavior of the learning gain. The main contribution of this paper is the study of a deterministic discrete-time (DDT) formulation that characterizes the average evolution of the node, preserving the discrete-time form of the original network and gathering a more realistic behavior of the learning gain. The new deterministic discrete-time model provides some unstability results (critical for the case of large similar variance signals) which are drastically different to the ones known for the continuous-time formulation. Simulation examples support the presented results, illustrating the practical limitations of the basic Hebbian model. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning in the multiple class random neural network

    Page(s): 1257 - 1267
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (746 KB)  

    Spiked recurrent neural networks with "multiple classes" of signals have been recently introduced by Gelenbe and Fourneau (1999), as an extension of the recurrent spiked random neural network introduced by Gelenbe (1989). These new networks can represent interconnected neurons, which simultaneously process multiple streams of data such as the color information of images, or networks which simultaneously process streams of data from multiple sensors. This paper introduces a learning algorithm which applies both to recurrent and feedforward multiple signal class random neural networks (MCRNNs). It is based on gradient descent optimization of a cost function. The algorithm exploits the analytical properties of the MCRNN and requires the solution of a system of nC linear and nC nonlinear equations (where C is the number of signal classes and n is the number of neurons) each time the network learns a new input-output pair. Thus, the algorithm is of O([nC]3) complexity for the recurrent case, and O([nC]2) for a feedforward MCRNN. Finally, we apply this learning algorithm to color texture modeling (learning), based on learning the weights of a recurrent network directly from the color texture image. The same trained recurrent network is then used to generate a synthetic texture that imitates the original. This approach is illustrated with various synthetic and natural textures. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recursive least-squares backpropagation algorithm for stop-and-go decision-directed blind equalization

    Page(s): 1472 - 1481
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (903 KB)  

    Stop-and-go decision-directed (S&G-DD) equalization is the most primitive blind equalization (BE) method for the cancelling of intersymbol-interference in data communication systems. Recently, this scheme has been applied to complex-valued multilayer feedforward neural network, giving robust results with a lower mean-square error at the expense of slow convergence. To overcome this problem, in this work, a fast converging recursive least squares (RLS)-based complex-valued backpropagation learning algorithm is derived for S&G-DD blind equalization. Simulation results show the effectiveness of the proposed algorithm in terms of initial convergence. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Deterministic bit-stream digital neurons

    Page(s): 1514 - 1525
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2408 KB) |  | HTML iconHTML  

    In this paper, we present the design of a deterministic bit-stream neuron, which makes use of the memory rich architecture of fine-grained field-programmable gate arrays (FPGAs). It is shown that deterministic bit streams provide the same accuracy as much longer stochastic bit streams. As these bit streams are processed serially, this allows neurons to be implemented that are much faster than those that utilize stochastic logic. Furthermore, due to the memory rich architecture of fine-grained FPGAs, these neurons still require only a small amount of logic to implement. The design presented here has been implemented on a Virtex FPGA, which allows a very regular layout facilitating efficient usage of space. This allows for the construction of neural networks large enough to solve complex tasks at a speed comparable to that provided by commercially available neural-network hardware. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Statistical analysis of the parameters of a neuro-genetic algorithm

    Page(s): 1374 - 1394
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (768 KB) |  | HTML iconHTML  

    Interest in hybrid methods that combine artificial neural networks and evolutionary algorithms has grown in the last few years, due to their robustness and ability to design networks by setting initial weight values, by searching the architecture and the learning rule and parameters. This paper presents an exhaustive analysis of the G-Prop method, and the different parameters the method requires (population size, selection rate, initial weight range, number of training epochs, etc.) are determined. The paper also the discusses the influence of the application of genetic operators on the precision (classification ability or error) and network size in classification problems. The significance and relative importance of the parameters with respect to the results obtained, as well as suitable values for each, were obtained using the ANOVA (analysis of the variance). Experiments show the significance of parameters concerning the neural network and learning in the hybrid methods. The parameters found using this method were used to compare the G-Prop method both to itself with other parameter settings, and to other published methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Silicon synaptic adaptation mechanisms for homeostasis and contrast gain control

    Page(s): 1497 - 1503
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (367 KB) |  | HTML iconHTML  

    We explore homeostasis in a silicon integrate-and-fire neuron. The neuron adapts its firing rate over time periods on the order of seconds or minutes so that it returns to its spontaneous firing rate after a sustained perturbation. Homeostasis is implemented via two schemes. One scheme looks at the presynaptic activity and adapts the synaptic weight depending on the presynaptic spiking rate. The second scheme adapts the synaptic "threshold" depending on the neuron's activity. The threshold is lowered if the neuron's activity decreases over a long time and is increased for prolonged increase in postsynaptic activity. The presynaptic adaptation mechanism models the contrast adaptation responses observed in simple cortical cells. To obtain the long adaptation timescales we require, we used floating-gates. Otherwise, the capacitors we would have to use would be of such a size that we could not integrate them and so we could not incorporate such long-time adaptation mechanisms into a very large-scale integration (VLSI) network of neurons. The circuits for the adaptation mechanisms have been implemented in a 2-μm double-poly CMOS process with a bipolar option. The results shown here are measured from a chip fabricated in this process. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability of fully asynchronous discrete-time discrete-state dynamic networks

    Page(s): 1353 - 1363
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (718 KB) |  | HTML iconHTML  

    We consider networks of a large number of neurons (or units, processors, ...), whose dynamics are fully asynchronous with overlapping updating. We suppose that the neurons take a finite number of states (discrete states), and that the updating scheme is discrete in time. We make no hypotheses on the activation function of the neurons; the networks may have multiple cycles and basins. We derive conditions on the initialization of the networks, which ensures convergence to fixed points only. Application to a fully asynchronous Hopfield neural network allows us to validate our study. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An ART-based construction of RBF networks

    Page(s): 1308 - 1321
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1069 KB) |  | HTML iconHTML  

    Radial basis function (RBF) networks are widely used for modeling a function from given input-output patterns. However, two difficulties are involved with traditional RBF (TRBF) networks: The initial configuration of an RBF network needs to be determined by a trial-and-error method, and the performance suffers when the desired output has abrupt changes or constant values in certain intervals. We propose a novel approach to over. come these difficulties. New kernel functions are used for hidden nodes, and the number of nodes is determined automatically by an adaptive resonance theory (ART)-like algorithm. Parameters and weights are initialized appropriately, and then tuned and adjusted by the gradient-descent method to improve the performance of the network. Experimental results have shown that the RBF networks constructed by our method have a smaller number of nodes, a faster learning speed, and a smaller approximation error than the networks produced by other methods. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Gaussian activation functions using Markov chains

    Page(s): 1465 - 1471
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (436 KB)  

    We extend, in two major ways, earlier work in which sigmoidal neural nonlinearities were implemented using stochastic counters. 1) We define the signal to noise limitations of unipolar and bipolar stochastic arithmetic and signal processing. 2) We generalize the use of stochastic counters to include neural transfer functions employed in Gaussian mixture models. The hardware advantages of (nonlinear) stochastic signal processing (SSP) may be offset by increased processing time; we quantify these issues. The ability to realize accurate Gaussian activation functions for neurons in pulsed digital networks using simple hardware with stochastic signals is also analyzed quantitatively. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust support vector regression networks for function approximation with outliers

    Page(s): 1322 - 1330
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (533 KB) |  | HTML iconHTML  

    Support vector regression (SVR) employs the support vector machine (SVM) to tackle problems of function approximation and regression estimation. SVR has been shown to have good robust properties against noise. When the parameters used in SVR are improperly selected, overfitting phenomena may still occur. However, the selection of various parameters is not straightforward. Besides, in SVR, outliers may also possibly be taken as support vectors. Such an inclusion of outliers in support vectors may lead to seriously overfitting phenomena. In this paper, a novel regression approach, termed as the robust support vector regression (RSVR) network, is proposed to enhance the robust capability of SVR. In the approach, traditional robust learning approaches are employed to improve the learning performance for any selected parameters. From the simulation results, our RSVR can always improve the performance of the learned systems for all cases. Besides, it can be found that even the training lasted for a long period, the testing errors would not go up. In other words, the overfitting phenomenon is indeed suppressed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Region growing with pulse-coupled neural networks: an alternative to seeded region growing

    Page(s): 1557 - 1562
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (757 KB)  

    The seeded region growing (SRG) algorithm is a fast robust parameter-free method for segmenting intensity images given initial seed locations for each region. The requirement of predetermined seeds means that the model cannot operate fully autonomously. In this paper, we demonstrate a novel region growing variant of the pulse-coupled neural network (PCNN), which offers comparable performance to the SRG and is able to generate seed locations internally, opening the way to fully autonomous operation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Deterministic nonmonotone strategies for effective training of multilayer perceptrons

    Page(s): 1268 - 1284
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1030 KB) |  | HTML iconHTML  

    We present deterministic nonmonotone learning strategies for multilayer perceptrons (MLPs), i.e., deterministic training algorithms in which error function values are allowed to increase at some epochs. To this end, we argue that the current error function value must satisfy a nonmonotone criterion with respect to the maximum error function value of the M previous epochs, and we propose a subprocedure to dynamically compute M. The nonmonotone strategy can be incorporated in any batch training algorithm and provides fast, stable, and reliable learning. Experimental results in different classes of problems show that this approach improves the convergence speed and success percentage of first-order training algorithms and alleviates the need for fine-tuning problem-depended heuristic parameters. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A finite-element mesh generator based on growing neural networks

    Page(s): 1482 - 1496
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2897 KB) |  | HTML iconHTML  

    A mesh generator for the production of high-quality finite-element meshes is being proposed. The mesh generator uses an artificial neural network, which grows during the training process in order to adapt itself to a prespecified probability distribution. The initial mesh is a constrained Delaunay triangulation of the domain to be triangulated. Two new algorithms to accelerate the location of the best matching unit are introduced. The mesh generator has been found able to produce meshes of high quality in a number of classic cases examined and is highly suited for problems where the mesh density vector can be calculated in advance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Chaotic simulated annealing with decaying chaotic noise

    Page(s): 1526 - 1531
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (342 KB) |  | HTML iconHTML  

    By adding chaotic noise to each neuron of the discrete-time continuous-output Hopfield neural network (HNN) and gradually reducing the noise, a chaotic neural network is proposed so that it is initially chaotic but eventually convergent, and, thus, has richer and more flexible dynamics compared to the HNN. The proposed network is applied to the traveling salesman problem (TSP) and that results are highly satisfactory. That is, the transient chaos enables the network to escape from local energy minima and to find global minima in 100% of the simulations for four-city and ten-city TSPs, as well as near-optimal solutions in most of runs for a 48-city TSP. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A decentralized control of interconnected systems using neural networks

    Page(s): 1554 - 1557
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (312 KB) |  | HTML iconHTML  

    We develop a decentralized neural-network (NN) controller for a class of large-scale nonlinear systems with the high-order interconnections. The controller is a mixed NN comprised of a conventional NN and a special NN. The conventional NN is used to approximate the unknown nonlinearities in the subsystem, while a special NN is used to counter the high-order interconnections. We prove that this NN structure can achieve a stable controller for the large-scale systems. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalized neural trees for pattern classification

    Page(s): 1540 - 1547
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (655 KB) |  | HTML iconHTML  

    In this paper, a new neural tree (NT) model, the generalized NT (GNT), is presented. The main novelty of the GNT consists in the definition of a new training rule that performs an overall optimization of the tree. Each time the tree is increased by a new level, the whole tree is reevaluated. The training rule uses a weight correction strategy that takes into account the entire tree structure, and it applies a normalization procedure to the activation values of each node such that these values can be interpreted as a probability. The weight connection updating is calculated by minimizing a cost function, which represents a measure of the overall probability of correct classification. Significant results on both synthetic and real data have been obtained by comparing the classification performances among multilayer perceptrons (MLPs), NTs, and GNTs. In particular, the GNT model displays good classification performances for training sets having complex distributions. Moreover, its particular structure provides an easily probabilistic interpretation of the pattern classification task and allows growing small neural trees with good generalization properties. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Neural Networks is devoted to the science and technology of neural networks, which disclose significant technical knowledge, exploratory developments, and applications of neural networks from biology to software to hardware.

 

This Transactions ceased production in 2011. The current retitled publication is IEEE Transactions on Neural Networks and Learning Systems.

Full Aims & Scope