http://ieeexplore.ieee.org TOC Alert for Publication# 3477 2008 July 24 This paper describes several ensemble methods that combine multiple different reinforcement learning (RL) algorithms in a single agent. The aim is to enhance learning speed and final performance by combining the chosen actions or action probabilities of different RL algorithms. We designed and implemented four different ensemble methods combining the following five different RL algorithms: $Q$ -learning, Sarsa, actor–critic (AC), $QV$-learning, and AC learning automaton. The intuitively designed ensemble methods, namely, majority voting (MV), rank voting, Boltzmann multiplication (BM), and Boltzmann addition, combine the policies derived from the value functions of the different RL algorithms, in contrast to previous work where ensemble methods have been used in RL for representing and learning a single value function. We show experiments on five maze problems of varying complexity; the first problem is simple, but the other four maze tasks are of a dynamic or partially observable nature. The results indicate that the BM and MV ensembles significantly outperform the single RL algorithms. ]]> 38 4 930 936 252 An evolutionary approach toward automatic generation of fuzzy inference systems (FISs), termed evolutionary dynamic self-generated fuzzy inference systems (EDSGFISs), is proposed in this paper. The structure and parameters of an FIS are generated through reinforcement learning, whereas an action set for training the consequents of the FIS is evolved via genetic algorithms (GAs). The proposed EDSGFIS algorithm can automatically create, delete, and adjust fuzzy rules according to the performance of the entire system, as well as evaluation of individual fuzzy rules. Simulation studies on a wall-following task by a mobile robot show that the proposed EDSGFIS approach is superior to other related methods. ]]> 38 4 963 969 473 This paper proposes algorithms for iris segmentation, quality enhancement, match score fusion, and indexing to improve both the accuracy and the speed of iris recognition. A curve evolution approach is proposed to effectively segment a nonideal iris image using the modified Mumford–Shah functional. Different enhancement algorithms are concurrently applied on the segmented iris image to produce multiple enhanced versions of the iris image. A support-vector-machine-based learning algorithm selects locally enhanced regions from each globally enhanced image and combines these good-quality regions to create a single high-quality iris image. Two distinct features are extracted from the high-quality iris image. The global textural feature is extracted using the 1-D log polar Gabor transform, and the local topological feature is extracted using Euler numbers. An intelligent fusion algorithm combines the textural and topological matching scores to further improve the iris recognition performance and reduce the false rejection rate, whereas an indexing algorithm enables fast and accurate iris identification. The verification and identification performance of the proposed algorithms is validated and compared with other algorithms using the CASIA Version 3, ICE 2005, and UBIRIS iris databases. ]]> 38 4 1021 1035 1283 This correspondence paper focuses on the problem of exponential stability for neural networks with a time-varying delay. The relationship among the time-varying delay, its upper bound, and their difference is taken into account. As a result, an improved linear-matrix-inequality-based delay-dependent exponential stability criterion is obtained without ignoring any terms in the derivative of Lyapunov–Krasovskii functional. Two numerical examples are given to demonstrate its effectiveness. ]]> 38 4 1152 1156 208 In genetic programming (GP), evolving tree nodes separately would reduce the huge solution space. However, tree nodes are highly interdependent with respect to their fitness. In this paper, we propose a new GP framework, namely, instruction-matrix (IM)-based GP (IMGP), to handle their interactions. IMGP maintains an IM to evolve tree nodes and subtrees separately. IMGP extracts program trees from an IM and updates the IM with the information of the extracted program trees. As the IM actually keeps most of the information of the schemata of GP and evolves the schemata directly, IMGP is effective and efficient. Our experimental results on benchmark problems have verified that IMGP is not only better than those of canonical GP in terms of the qualities of the solutions and the number of program evaluations, but they are also better than some of the related GP algorithms. IMGP can also be used to evolve programs for classification problems. The classifiers obtained have higher classification accuracies than four other GP classification algorithms on four benchmark classification problems. The testing errors are also comparable to or better than those obtained with well-known classifiers. Furthermore, an extended version, called condition matrix for rule learning, has been used successfully to handle multiclass classification problems. ]]> 38 4 1036 1049 530 Learning automata (LA) were recently shown to be valuable tools for designing multiagent reinforcement learning algorithms. One of the principal contributions of the LA theory is that a set of decentralized independent LA is able to control a finite Markov chain with unknown transition probabilities and rewards. In this paper, we propose to extend this algorithm to Markov games—a straightforward extension of single-agent Markov decision problems to distributed multiagent decision problems. We show that under the same ergodic assumptions of the original theorem, the extended algorithm will converge to a pure equilibrium point between agent policies. ]]> 38 4 976 981 267 An adaptive critic design (ACD)-based damping controller is developed for a thyristor-controlled series capacitor (TCSC) installed in a power system with multiple poorly damped interarea modes. The performance of this ACD computational intelligence-based method is compared with two classical techniques, which are observer-based state-feedback (SF) control and linear matrix inequality $hbox{LMI-H}^{infty}$ robust control. Remote measurements are used as feedback signals to the wide-area damping controller for modulating the compensation of the TCSC. The classical methods use a linearized model of the system whereas the ACD method is purely measurement-based, leading to a nonlinear controller with fixed parameters. A comparative analysis of the controllers' performances is carried out under different disturbance scenarios. The ACD-based design has shown promising performance with very little knowledge of the system compared to classical model-based controllers. This paper also discusses the advantages and disadvantages of ACDs, SF, and $hbox{LMI-H}^{infty}$. ]]> 38 4 1002 1007 689 This correspondence paper is concerned with the problem of designing switched dynamic output feedback $H_{infty}$ controllers for continuous-time Takagi–Sugeno (T–S) fuzzy systems. A new type of dynamic output feedback controllers, namely, switched dynamic parallel distributed compensation (SDPDC) controllers, is proposed, which are switched by basing on the values of membership functions. A new method for designing SDPDC controllers for guaranteeing stabilities and $H_{infty}$ performances of closed-loop nonlinear systems is presented, where the design conditions are given in terms of the solvability of a set of linear matrix inequalities. It is shown that the new method provides better or at least the same results of the existing design methods via a pure DPDC scheme. A numerical example is given to illustrate the effectiveness of the proposed method. ]]> 38 4 1166 1175 692 In this paper, an incoherent control scheme for accomplishing the state control of a class of quantum systems which have wavefunction-controllable subspaces is proposed. This scheme includes the following two steps: projective measurement on the initial state and learning control in the wavefunction-controllable subspace. The first step probabilistically projects the initial state into the wavefunction-controllable subspace. The probability of success is sensitive to the initial state; however, it can be greatly improved through multiple experiments on several identical initial states even in the case with a small probability of success for an individual measurement. The second step finds a local optimal control sequence via quantum reinforcement learning and drives the controlled system to the objective state through a set of suitable controls. In this strategy, the initial states can be unknown identical states, the quantum measurement is used as an effective control, and the controlled system is not necessarily unitarily controllable. This incoherent control scheme provides an alternative quantum engineering strategy for locally controllable quantum systems. ]]> 38 4 957 962 572 Two distinguishing features of humanlike control vis-À-vis current technological control are the ability to make use of experience while selecting a control policy for distinct situations and the ability to do so faster and faster as more experience is gained (in contrast to current technological implementations that slow down as more knowledge is stored). The notions of context and context discernment are important to understanding this human ability. Whereas methods known as adaptive control and learning control focus on modifying the design of a controller as changes in context occur, experience-based (EB) control entails selecting a previously designed controller that is appropriate to the current situation. Developing the EB approach entails a shift of the technologist's focus “up a level” away from designing individual (optimal) controllers to that of developing online algorithms that efficiently and effectively select designs from a repository of existing controller solutions. A key component of the notions presented here is that of higher level learning algorithm. This is a new application of reinforcement learning and, in particular, approximate dynamic programming, with its focus shifted to the posited higher level, and is employed, with very promising results. The author's hope for this paper is to inspire and guide future work in this promising area. ]]> 38 4 901 912 604 A new approach for engine calibration and control is proposed. In this paper, we present our research results on the implementation of adaptive critic designs for self-learning control of automotive engines. A class of adaptive critic designs that can be classified as (model-free) action-dependent heuristic dynamic programming is used in this research project. The goals of the present learning control design for automotive engines include improved performance, reduced emissions, and maintained optimum performance under various operating conditions. Using the data from a test vehicle with a V8 engine, we developed a neural network model of the engine and neural network controllers based on the idea of approximate dynamic programming to achieve optimal control. We have developed and simulated self-learning neural network controllers for both engine torque (TRQ) and exhaust air–fuel ratio (AFR) control. The goal of TRQ control and AFR control is to track the commanded values. For both control problems, excellent neural network controller transient performance has been achieved. ]]> 38 4 988 993 212 This paper applies a neural-network-based approximate dynamic programming method, namely, the direct heuristic dynamic programming (direct HDP), to a large power system stability control problem. The direct HDP is a learning- and approximation-based approach to addressing nonlinear coordinated control under uncertainty. One of the major design parameters, the controller learning objective function, is formulated to directly account for network-wide low-frequency oscillation with the presence of nonlinearity, uncertainty, and coupling effect among system components. Results include a novel learning control structure based on the direct HDP with applications to two power system problems. The first case involves static var compensator supplementary damping control, which is used to provide a comprehensive evaluation of the learning control performance. The second case aims at addressing a difficult complex system challenge by providing a new solution to a large interconnected power network oscillation damping control problem that frequently occurs in the China Southern Power Grid. ]]> 38 4 1008 1013 271 Fast training and testing procedures are crucial in biometrics recognition research. Conventional algorithms, e.g., principal component analysis (PCA), fail to efficiently work on large-scale and high-resolution image data sets. By incorporating merits from both two-dimensional PCA (2DPCA)-based image decomposition and fast numerical calculations based on Haarlike bases, this technical correspondence first proposes binary 2DPCA (B-2DPCA). Empirical studies demonstrated the advantages of B-2DPCA compared with 2DPCA and binary PCA. ]]> 38 4 1176 1180 363 Convergence of the value-iteration-based heuristic dynamic programming (HDP) algorithm is proven in the case of general nonlinear systems. That is, it is shown that HDP converges to the optimal control and the optimal value function that solves the Hamilton–Jacobi–Bellman equation appearing in infinite-horizon discrete-time (DT) nonlinear optimal control. It is assumed that, at each iteration, the value and action update equations can be exactly solved. The following two standard neural networks (NN) are used: a critic NN is used to approximate the value function, whereas an action network is used to approximate the optimal control policy. It is stressed that this approach allows the implementation of HDP without knowing the internal dynamics of the system. The exact solution assumption holds for some classes of nonlinear systems and, specifically, in the specific case of the DT linear quadratic regulator (LQR), where the action is linear and the value quadratic in the states and NNs have zero approximation error. It is stressed that, for the LQR, HDP may be implemented without knowing the system $A$ matrix by using two NNs. This fact is not generally appreciated in the folklore of HDP for the DT LQR, where only one critic NN is generally used. ]]> 38 4 943 949 233 This paper traces the development of neural-network (NN)-based feedback controllers that are derived from the principle of adaptive/approximate dynamic programming (ADP) and discusses their closed-loop stability. Different versions of NN structures in the literature, which embed mathematical mappings related to solutions of the ADP-formulated problems called “adaptive critics” or “action-critic” networks, are discussed. Distinction between the two classes of ADP applications is pointed out. Furthermore, papers in “model-free” development and model-based neurocontrollers are reviewed in terms of their contributions to stability issues. Recent literature suggests that work in ADP-based feedback controllers with assured stability is growing in diverse forms. ]]> 38 4 913 917 145 This correspondence presents an approach to the detection and isolation of component failures in large-scale systems. In the case of sensors that report at rates of 1 Hz or less, the algorithm can be considered real time. The input is a set of observed test results from multiple sensors, and the algorithm's main task is to deal with sensor errors. The sensors are assumed to be of threshold test (pass/fail) type, but to be vulnerable to noise, in that occasionally true failures are missed, and likewise, there can be false alarms. These errors are further assumed to be independent conditioned on the system's diagnostic state. Their probabilities, of missed detection and of false alarm, are not known a priori and must be estimated (ideally along with the accuracies of these estimates) online, within the inference engine. Further, recognizing a practical concern in most real systems, a sparsely instantiated observation vector must not be a problem. The key ingredients to our solution include the multiple-hypothesis tracking philosophy to complexity management, a Beta prior distribution on the sensor errors, and a quickest detection overlay to detect changes in these error rates when the prior is violated. We provide results illustrating performance in terms of both computational needs and error rate, and show its application both as a filter (i.e., used to “clean” sensor reports) and as a standalone state estimator. ]]> 38 4 1157 1165 817 The time scales calculus is a key emerging area of mathematics due to its potential use in a wide variety of multidisciplinary applications. We extend this calculus to approximate dynamic programming (ADP). The core backward induction algorithm of dynamic programming is extended from its traditional discrete case to all isolated time scales. Hamilton–Jacobi–Bellman equations, the solution of which is the fundamental problem in the field of dynamic programming, are motivated and proven on time scales. By drawing together the calculus of time scales and the applied area of stochastic control via ADP, we have connected two major fields of research. ]]> 38 4 918 923 236 This paper presents an improved Adaptive–Reinforcement Learning Control methodology for the problem of unmanned air vehicle morphing control. The reinforcement learning morphing control function that learns the optimal shape change policy is integrated with an adaptive dynamic inversion control trajectory tracking function. An episodic unsupervised learning simulation using the Q-learning method is developed to replace an earlier and less accurate Actor-Critic algorithm. Sequential Function Approximation, a Galerkin-based scattered data approximation scheme, replaces a K-Nearest Neighbors (KNN) method and is used to generalize the learning from previously experienced quantized states and actions to the continuous state-action space, all of which may not have been experienced before. The improved method showed smaller errors and improved learning of the optimal shape compared to the KNN. ]]> 38 4 1014 1020 497 In this paper, we aim to solve the infinite-time optimal tracking control problem for a class of discrete-time nonlinear systems using the greedy heuristic dynamic programming (HDP) iteration algorithm. A new type of performance index is defined because the existing performance indexes are very difficult in solving this kind of tracking problem, if not impossible. Via system transformation, the optimal tracking problem is transformed into an optimal regulation problem, and then, the greedy HDP iteration algorithm is introduced to deal with the regulation problem with rigorous convergence analysis. Three neural networks are used to approximate the performance index, compute the optimal control policy, and model the nonlinear system for facilitating the implementation of the greedy HDP iteration algorithm. An example is given to demonstrate the validity of the proposed optimal tracking control scheme. ]]> 38 4 937 942 385 The control of dynamic systems that undergo an impact collision is both theoretically challenging and of practical importance. An appeal of studying systems that undergo an impact is that short-duration effects such as high stresses, rapid dissipation of energy, and fast acceleration and deceleration may be achieved from low-energy sources. However, colliding systems present a difficult control challenge because the equations of motion are different when the system suddenly transitions from a noncontact state to a contact state. In this paper, an adaptive nonlinear controller is designed to regulate the states of two dynamic systems that collide. The academic example of a planar robot colliding with an unactuated mass–spring system is used to represent a broader class of such systems. The control objective is defined as the desire to command a robot to collide with an unactuated system and regulate the mass to a desired compressed state while compensating for the unknown constant system parameters. Lyapunov-based methods are used to develop a continuous adaptive controller that yields asymptotic regulation of the mass and robot links. It is interesting to note that one controller is responsible for achieving the control objective when the robot is in free motion (i.e., decoupled from the mass–spring system), when the systems collide, and when the system dynamics are coupled. ]]> 38 4 1050 1061 532 A constrained approximate dynamic programming (ADP) approach is presented for designing adaptive neural network (NN) controllers with closed-loop stability and performance guarantees. Prior knowledge of the linearized equations of motion is used to guarantee that the closed-loop system meets performance and stability objectives when the plant operates in a linear parameter-varying (LPV) regime. In the presence of unmodeled dynamics or failures, the NN controller adapts to optimize its performance online, whereas constrained ADP guarantees that the LPV baseline performance is preserved at all times. The effectiveness of an adaptive NN flight controller is demonstrated for simulated control failures, parameter variations, and near-stall dynamics. ]]> 38 4 982 987 274 In this paper, the synchronization control problem is considered for two coupled discrete-time complex networks with time delays. The network under investigation is quite general to reflect the reality, where the state delays are allowed to be time varying with given lower and upper bounds, and the stochastic disturbances are assumed to be Brownian motions that affect not only the network coupling but also the overall networks. By utilizing the Lyapunov functional method combined with linear matrix inequality (LMI) techniques, we obtain several sufficient delay-dependent conditions that ensure the coupled networks to be globally exponentially synchronized in the mean square. A control law is designed to synchronize the addressed coupled complex networks in terms of certain LMIs that can be readily solved using the Matlab LMI toolbox. Two numerical examples are presented to show the validity of our theoretical analysis results. ]]> 38 4 1073 1083 377 Multiagent systems are inherently associated with their distributivity, which enforces a great deal of communication mechanisms. To effectively arrive at meaningful solutions in a vast array of problem-solving tasks, it becomes imperative to establish a sound machinery of reconciling findings which might form partial solutions to an overall problem. In this paper, we focus on a broad category of problems of collaborative data analysis realized by a collection of agents having access to their individual data and exchanging findings through their collaboration activities. Such problems of data analysis arise in the context of building a global view at a certain phenomenon (process) by viewing it from different perspectives (and thus engaging various collections of attributes by various agents). Our goal is to develop some interaction between the agents so that they could form an overall perspective, where the knowledge available locally is shared and reconciled. The underlying format of knowledge built by the agents is that of information granules and fuzzy sets in particular. We develop a comprehensive optimization scheme and discuss its two-phase nature in which the communication phase of the granular findings intertwines with the local optimization being realized by the agents at the level of the individual datasite and exploits the evidence collected from other sites. We show how the mechanism of fuzzy granulation realized in the form of a well-known fuzzy c-means (FCM) clustering can be augmented to support collaborative activities required by the agents. For this purpose, we introduce augmented versions of the original objective function used in the FCM and derive algorithmic details. We also discuss an issue of optimizing the strength of collaborative linkages, so that the reconciled findings attain the highest level of consistency (agreement). The presented experimental studies include some synthetic data and selected data sets coming from the Machine L- earning repository. ]]> 38 4 1062 1072 883 We combine three threads of research on approximate dynamic programming: sparse random sampling of states, value function and policy approximation using local models, and using local trajectory optimizers to globally optimize a policy and associated value function. Our focus is on finding steady-state policies for deterministic time-invariant discrete time control problems with continuous states and actions often found in robotics. In this paper, we describe our approach and provide initial results on several simulated robotics problems. ]]> 38 4 924 929 410 Many interesting problems in reinforcement learning (RL) are continuous and/or high dimensional, and in this instance, RL techniques require the use of function approximators for learning value functions and policies. Often, local linear models have been preferred over distributed nonlinear models for function approximation in RL. We suggest that one reason for the difficulties encountered when using distributed architectures in RL is the problem of negative interference, whereby learning of new data disrupts previously learned mappings. The continuous temporal difference (TD) learning algorithm $TD(lambda)$ was used to learn a value function in a limited-torque pendulum swing-up task using a multilayer perceptron (MLP) network. Three different approaches were examined for learning in the MLP networks; 1) simple gradient descent; 2) vario-eta; and 3) a pseudopattern rehearsal strategy that attempts to reduce the effects of interference. Our results show that MLP networks can be used for value function approximation in this task but require long training times. We also found that vario-eta destabilized learning and resulted in a failure of the learning process to converge. Finally, we showed that the pseudopattern rehearsal strategy drastically improved the speed of learning. The results indicate that interference is a greater problem than ill conditioning for this task. ]]> 38 4 950 956 645 This paper introduce a type-2 fuzzy function system for uncertainty modeling using evolutionary algorithms (ET2FF). The type-1 fuzzy inference systems (FISs) with fuzzy functions, which do not entail if $ldots$ then rule bases, have demonstrated better performance compared to traditional FIS. Nonetheless, the performance of these approaches is usually affected by their uncertain parameters. The proposed method implements a three-phase learning strategy to capture the uncertainties in fuzzy function systems induced by learning parameters, as well as fuzzy function structures. The improved fuzzy clustering initially finds hidden structures, and the genetic learning algorithm optimizes interval type-2 fuzzy sets to capture their optimum uncertainty interval. The proposed ET2FF architecture is evaluated using an extensive suite of real-life applications such as manufacturing process and financial market modeling. The results show that the proposed ET2FF method is comparable—if not superior—to earlier FIS in terms of generalization performance and robustness. ]]> 38 4 1098 1110 1284 38 4 C1 895 164 38 4 C2 C2 38 38 4 C3 C3 28 38 4 C4 C4 33 38 4 898 900 199 38 4 896 897 129 This paper investigates the stability and boundary of a swarm with a general directed and weighted topology. The stability of a swarm is generally considered as cohesiveness. We construct a symmetric eigenmatrix and define an orthogonal eigenparameter which reflects the degree of orthogonality between the left eigenvector of the coupling matrix corresponding to its zero eigenvalue and the eigenvectors of the eigenmatrix corresponding to its nonzero eigenvalues. We prove that, if the topology of the underlying swarm is strongly connected, the swarm is then stable in the sense that all agents will globally and exponentially converge to a hyperellipsoid in finite time, both in open space and profiles, whether the center of the hyperellipsoid is moving or not. The swarm boundary and convergence rate are characterized by the eigenparameters of the swarm, which reveals the quantitative relationship between the swarming behavior and characteristics of the coupling topology. ]]> 38 4 1084 1097 1196 A new method, which is called decomposition–composition (D–C) method, is proposed for the nonlinear dimensionality reduction (NLDR) of data lying on the multicluster manifold. The main idea is first to decompose a given data set into clusters and independently calculate the low-dimensional embeddings of each cluster by the decomposition procedure. Based on the intercluster connections, the embeddings of all clusters are then composed into their proper positions and orientations by the composition procedure. Different from other NLDR methods for multicluster data, which consider associatively the intracluster and intercluster information, the D–C method capitalizes on the separate employment of the intracluster neighborhood structures and the intercluster topologies for effective dimensionality reduction. This, on one hand, isometrically preserves the rigid-body shapes of the clusters in the embedding process and, on the other hand, guarantees the proper locations and orientations of all clusters. The theoretical arguments are supported by a series of experiments performed on the synthetic and real-life data sets. In addition, the computational complexity of the proposed method is analyzed, and its efficiency is theoretically analyzed and experimentally demonstrated. Related strategies for automatic parameter selection are also examined. ]]> 38 4 1111 1122 1442 38 4 1181 1181 203 38 4 1182 1182 354 For the interaction of a mobile robot with a dynamic environment, the estimation of object motion is desired while the robot is walking and/or turning its head. In this paper, we describe a system which manages this task by combining depth from a stereo camera and computation of the camera movement from robot kinematics in order to stabilize the camera images. Moving objects are detected by applying optical flow to the stabilized images followed by a filtering method, which incorporates both prior knowledge about the accuracy of the measurement and the uncertainties of the measurement process itself. The efficiency of this system is demonstrated in a dynamic real-world scenario with a walking humanoid robot. ]]> 38 4 1139 1151 1248 The design of robust and high-performance gaze-tracking systems is one of the most important objectives of the eye-tracking community. In general, a subject calibration procedure is needed to learn system parameters and be able to estimate the gaze direction accurately. In this paper, we attempt to determine if subject calibration can be eliminated. A geometric analysis of a gaze-tracking system is conducted to determine user calibration requirements. The eye model used considers the offset between optical and visual axes, the refraction of the cornea, and Donder's law. This paper demonstrates the minimal number of cameras, light sources, and user calibration points needed to solve for gaze estimation. The underlying geometric model is based on glint positions and pupil ellipse in the image, and the minimal hardware needed for this model is one camera and multiple light-emitting diodes. This paper proves that subject calibration is compulsory for correct gaze estimation and proposes a model based on a single point for subject calibration. The experiments carried out show that, although two glints and one calibration point are sufficient to perform gaze estimation (error $sim!!1^{circ}$ ), using more light sources and calibration points can result in lower average errors. ]]> 38 4 1123 1138 1386 A nonaffine discrete-time system represented by the nonlinear autoregressive moving average with eXogenous input (NARMAX) representation with unknown nonlinear system dynamics is considered. An equivalent affinelike representation in terms of the tracking error dynamics is first obtained from the original nonaffine nonlinear discrete-time system so that reinforcement-learning-based near-optimal neural network (NN) controller can be developed. The control scheme consists of two linearly parameterized NNs. One NN is designated as the critic NN, which approximates a predefined long-term cost function, and an action NN is employed to derive a near-optimal control signal for the system to track a desired trajectory while minimizing the cost function simultaneously. The NN weights are tuned online. By using the standard Lyapunov approach, the stability of the closed-loop system is shown. The net result is a supervised actor-critic NN controller scheme which can be applied to a general nonaffine nonlinear discrete-time system without needing the affinelike representation. Simulation results demonstrate satisfactory performance of the controller. ]]> 38 4 994 1001 336 We consider decentralized Bayesian search problems that involve a team of multiple autonomous agents searching for targets on a network of search points operating under the following constraints: 1) interagent communication is limited; 2) the agents do not have the opportunity to agree in advance on how to resolve equivalent but incompatible strategies; and 3) each agent lacks the ability to control or predict with certainty the actions of the other agents. We formulate the multiagent search-path-planning problem as a decentralized optimal control problem and introduce approximate dynamic heuristics that can be implemented in a decentralized fashion. After establishing some analytical properties of the heuristics, we present computational results for a search problem involving two agents on a 5 $times$ 5 grid. ]]> 38 4 970 975 310