By Topic

Evolutionary Computation, IEEE Transactions on

Issue 6 • Date Dec. 2005

Filter Results

Displaying Results 1 - 21 of 21
  • Table of contents

    Page(s): c1
    Save to Project icon | Request Permissions | PDF file iconPDF (39 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Evolutionary Computation publication information

    Page(s): c2
    Save to Project icon | Request Permissions | PDF file iconPDF (36 KB)  
    Freely Available from IEEE
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Behavioral diversity, choices and noise in the iterated prisoner's dilemma

    Page(s): 540 - 551
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (880 KB) |  | HTML iconHTML  

    Real-world dilemmas rarely involve just two choices and perfect interactions without mistakes. In the iterated prisoner's dilemma (IPD) game, intermediate choices or mistakes (noise) have been introduced to extend its realism. This paper studies the IPD game with both noise and multiple levels of cooperation (intermediate choices) in a coevolutionary environment, where players can learn and adapt their strategies through an evolutionary algorithm. The impact of noise on the evolution of cooperation is first examined. It is shown that the coevolutionary models presented in this paper are robust against low noise (when mistakes occur with low probability). That is, low levels of noise have little impact on the evolution of cooperation. On the other hand, high noise (when mistakes occur with high probability) creates misunderstandings and discourages cooperation. However, the evolution of cooperation in the IPD with more choices in a coevolutionary learning setting also depends on behavioral diversity. This paper further investigates the issue of behavioral diversity in the coevolution of strategies for the IPD with more choices and noise. The evolution of cooperation is more difficult to achieve if a coevolutionary model with low behavioral diversity is used for IPD games with higher levels of noise. The coevolutionary model with high behavioral diversity in the population is more resistant to noise. It is shown that strategy representations can have a significant impact on the evolutionary outcomes because of different behavioral diversities that they generate. The results further show the importance of behavioral diversity in coevolutionary learning. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Evolution of iterated prisoner's dilemma game strategies in structured demes under random pairing in game playing

    Page(s): 552 - 561
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (656 KB) |  | HTML iconHTML  

    We discuss the evolution of strategies in a spatial iterated prisoner's dilemma (IPD) game in which each player is located in a cell of a two-dimensional grid-world. Following the concept of structured demes, two neighborhood structures are used. One is for the interaction among players through the IPD game. A player in each cell plays against its neighbors defined by this neighborhood structure. The other is for mating strategies by genetic operations. A new strategy for a player is generated by genetic operations from a pair of parent strings, which are selected from its neighboring cells defined by the second neighborhood structure. After examining the effect of the two neighborhood structures on the evolution of cooperative behavior with standard pairing in game-playing, we introduce a random pairing scheme in which each player plays against a different randomly chosen neighbor at every round (i.e., every iteration) of the game. Through computer simulations, we demonstrate that small neighborhood structures facilitate the evolution of cooperative behavior under random pairing in game-playing. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Particle swarm optimization approaches to coevolve strategies for the iterated prisoner's dilemma

    Page(s): 562 - 579
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2448 KB) |  | HTML iconHTML  

    This paper presents and investigates the application of coevolutionary training techniques based on particle swarm optimization (PSO) to evolve playing strategies for the nonzero sum problem of the iterated prisoner's dilemma (IPD). Three different coevolutionary PSO techniques are used, differing in the way that IPD strategies are presented: A neural network (NN) approach in which the NN is used to predict the next action, a binary PSO approach in which the particle represents a complete playing strategy, and finally, a novel approach that exploits the symmetrical structure of man-made strategies. The last technique uses a PSO algorithm as a function approximator to evolve a function that characterizes the dynamics of the IPD. These different PSO approaches are compared experimentally with one another, and with popular man-made strategies. The performance of these approaches is evaluated in both clean and noisy environments. Results indicate that NNs cooperate well, but may develop weak strategies that can cause catastrophic collapses. The binary PSO technique does not have the same deficiency, instead resulting in an overall state of equilibrium in which some strategies are allowed to exploit the population, but never dominate. The symmetry approach is not as successful as the binary PSO approach in maintaining cooperation in both noisy and noiseless environments-exhibiting selfish behavior against the benchmark strategies and depriving them of receiving almost any payoff. Overall, the PSO techniques are successful at generating a variety of strategies for use in the IPD, duplicating and improving on existing evolutionary IPD population observations. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A game-theoretic and dynamical-systems analysis of selection methods in coevolution

    Page(s): 580 - 602
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2232 KB) |  | HTML iconHTML  

    We use evolutionary game theory (EGT) to investigate the dynamics and equilibria of selection methods in coevolutionary algorithms. The canonical selection method used in EGT is equivalent to the standard "fitness-proportional" selection method used in evolutionary algorithms. All attractors of the EGT dynamic are Nash equilibria; we focus on simple symmetric variable-sum games that have polymorphic Nash-equilibrium attractors. Against the dynamics of proportional selection, we contrast the behaviors of truncation selection, (μ,λ),(μ+λ), linear ranking, Boltzmann, and tournament selection. Except for Boltzmann selection, each of the methods we test unconditionally fail to achieve polymorphic Nash equilibrium. Instead, we find point attractors that lack game-theoretic justification, cyclic dynamics, or chaos. Boltzmann selection converges onto polymorphic Nash equilibrium only when selection pressure is sufficiently low; otherwise, we obtain attracting limit-cycles or chaos. Coevolutionary algorithms are often used to search for solutions (e.g., Nash equilibria) of games of strategy; our results show that many selection methods are inappropriate for finding polymorphic Nash solutions to variable-sum games. Another application of coevolution is to model other systems; our results emphasize the degree to which the model's behavior is sensitive to implementation details regarding selection-details that we might not otherwise believe to be critical. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Evolutionary asymmetric games for modeling systems of partially cooperative agents

    Page(s): 603 - 614
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (448 KB) |  | HTML iconHTML  

    Based on the tenet of Darwinism, we propose a general mechanism that guides agents (which can be partially cooperative) in selecting appropriate strategies in situations of complex interactions, in which agents do not have complete information about other agents. In the mechanism, each participating agent generates many instances of itself to help it find an appropriate strategy. The generated instances adopt alternative strategies from the agent's strategy set. While all instances generated by different agents meet randomly to complete a task, every instance adapts its strategy according to the difference between the average utilities of its current strategy and all its strategies. We give a complete analysis of the mechanism for the case with two agents when each agent has two strategies, and show that by the tenet of Darwinism, agents can find their appropriate strategies through evolution and adaptation: 1) if dominant strategies exist, then the proposed mechanism is guaranteed to find them; 2) if there are two or more strict Nash equilibrium strategies, the proposed mechanism is guaranteed to find them by using different initial strategy distributions; and 3) if there is no dominant strategy and no strict Nash equilibrium, then agents will oscillate periodically. Nevertheless, the mechanism allows agent designers to derive the appropriate strategies from the oscillation by integration. For cases with two agents when each agent has two or more strategies, it is shown that agents can reach a steady state where social welfare is optimum. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Systematically incorporating domain-specific knowledge into evolutionary speciated checkers players

    Page(s): 615 - 627
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1960 KB) |  | HTML iconHTML  

    The evolutionary approach for gaming is different from the traditional one that exploits knowledge of the opening, middle, and endgame stages. It is, therefore, sometimes inefficient to evolve simple heuristics that may be created easily by humans because it is based purely on a bottom-up style of construction. Incorporating domain knowledge into evolutionary computation can improve the performance of evolved strategies and accelerate the speed of evolution by reducing the search space. In this paper, we propose the systematic insertion of opening knowledge and an endgame database into the framework of evolutionary checkers. Also, the common knowledge that the combination of diverse strategies is better than a single best one is included in the middle stage and is implemented using crowding algorithm and a strategy combination scheme. Experimental results show that the proposed method is promising for generating better strategies. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Coevolution versus self-play temporal difference learning for acquiring position evaluation in small-board go

    Page(s): 628 - 640
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (472 KB) |  | HTML iconHTML  

    Two learning methods for acquiring position evaluation for small Go boards are studied and compared. In each case the function to be learned is a position-weighted piece counter and only the learning method differs. The methods studied are temporal difference learning (TDL) using the self-play gradient-descent method and coevolutionary learning, using an evolution strategy. The two approaches are compared with the hope of gaining a greater insight into the problem of searching for "optimal" zero-sum game strategies. Using tuned standard setups for each algorithm, it was found that the temporal-difference method learned faster, and in most cases also achieved a higher level of play than coevolution, providing that the gradient descent step size was chosen suitably. The performance of the coevolution method was found to be sensitive to the design of the evolutionary algorithm in several respects. Given the right configuration, however, coevolution achieved a higher level of play than TDL. Self-play results in optimal play against a copy of itself. A self-play player will prefer moves from which it is unlikely to lose even when it occasionally makes random exploratory moves. An evolutionary player forced to perform exploratory moves in the same way can achieve superior strategies to those acquired through self-play alone. The reason for this is that the evolutionary player is exposed to more varied game-play, because it plays against a diverse population of players. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Planning an endgame move set for the game RISK: a comparison of search algorithms

    Page(s): 641 - 652
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1440 KB) |  | HTML iconHTML  

    Seven algorithms used to search for solutions in dynamic planning and execution problems are compared. The specific problem is endgame moves for the board game RISK. This paper concentrates on comparison of search methods for the best plan using a fixed evaluation function, fixed time to plan, and randomly generated situations that correspond to endgames in RISK with eight remaining players. The search strategies compared are depth-first, breadth-first, best-first, random walk, gradient ascent, simulated annealing, and evolutionary computation. The approaches are compared for each example based on the number of opponents eliminated, plan completion probability, and value of ending position (if the moves do not complete the game). Simulation results indicate that the evolutionary approach is superior to the other methods in 85% of the cases considered. Among the other algorithms, simulated annealing is the most suitable for this problem. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Real-time neuroevolution in the NERO video game

    Page(s): 653 - 668
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2424 KB) |  | HTML iconHTML  

    In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. This paper introduces the real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises. To demonstrate this concept, the Neuroevolving Robotic Operatives (NERO) game was built based on rtNEAT. In NERO, the player trains a team of virtual robots for combat against other players' teams. This paper describes results from this novel application of machine learning, and demonstrates that rtNEAT makes possible video games like NERO where agents evolve and adapt in real time. In the future, rtNEAT may allow new kinds of educational and training applications through interactive and adapting games. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Playing to learn: case-injected genetic algorithms for learning to play computer games

    Page(s): 669 - 681
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1888 KB)  

    We use case-injected genetic algorithms (CIGARs) to learn to competently play computer strategy games. CIGARs periodically inject individuals that were successful in past games into the population of the GA working on the current game, biasing search toward known successful strategies. Computer strategy games are fundamentally resource allocation games characterized by complex long-term dynamics and by imperfect knowledge of the game state. CIGAR plays by extracting and solving the game's underlying resource allocation problems. We show how case injection can be used to learn to play better from a human's or system's game-playing experience and our approach to acquiring experience from human players showcases an elegant solution to the knowledge acquisition bottleneck in this domain. Results show that with an appropriate representation, case injection effectively biases the GA toward producing plans that contain important strategic elements from previously successful strategies. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Coevolutionary optimization of fuzzy logic intelligence for strategic decision support

    Page(s): 682 - 694
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (456 KB) |  | HTML iconHTML  

    We present a description and initial results of a computer code that coevolves fuzzy logic rules to play a two-sided zero-sum competitive game. It is based on the TEMPO Military Planning Game that has been used to teach resource allocation to over 20 000 students over the past 40 years. No feasible algorithm for optimal play is known. The coevolved rules, when pitted against human players, usually win the first few competitions. For reasons not yet understood, the evolved rules (found in a symmetrical competition) place little value on information concerning the play of the opponent. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Evolving assembly programs: how games help microprocessor validation

    Page(s): 695 - 706
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (792 KB) |  | HTML iconHTML  

    Core War is a game where two or more programs, called warriors, are executed in the same memory area by a time-sharing processor. The final goal of each warrior is to crash the others by overwriting them with illegal instructions. The game was popularized by A. K. Dewdney in his Scientific American column in the mid-1980s. In order to automatically devise strong warriors, μGP, a test program generation algorithm, was extended with the ability to assimilate existing code and to detect clones; furthermore, a new selection mechanism for promoting diversity independent from fitness calculations was added. The evolved warriors are the first machine-written programs ever able to become King of the Hill (champion) in all four main international Tiny Hills. This paper shows how playing Core War may help generate effective test programs for validation and test of microprocessors. Tackling a more mundane problem, the described techniques are currently being exploited for the automatic completion and refinement of existing test programs. Preliminary experimental results are reported. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Unraveling ancient mysteries: reimagining the past using evolutionary computation in a complex gaming environment

    Page(s): 707 - 720
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1848 KB) |  | HTML iconHTML  

    In this paper, we use principles from game theory, computer gaming, and evolutionary computation to produce a framework for investigating one of the great mysteries of the ancient Americas: why did the pre-Hispanic Pueblo (Anasazi) peoples leave large portions of their territories in the late A.D. 1200s? The gaming concept is overlaid on a large-scale agent-based simulation of the Anasazi. Agents in this game use a cultural algorithm framework to modify their finite-state automata (FSA) controllers following the work of Fogel (1966). In the game, there can be two kinds of active agents: scripted and unscripted. Unscripted agents attempt to maximize their survivability, whereas scripted agents can be used to test the impact that various pure and compound strategies for cooperation and defection have on the social structures produced by the overall system. The goal of our experiments here is to determine the extent to which cooperation and competition need to be present among the agent households in order to produce a population structure and spatial distribution similar to what has been observed archaeologically. We do this by embedding a "trust in networks" game within the simulation. In this game, agents can choose from three pure strategies: defect, trust, and inspect. This game does not have a pure Nash equilibrium but instead has a mixed strategy Nash equilibrium such that a certain proportion of the population uses each at every time step, where the proportion relates to the quality of the signal used by the inspectors to predict defection. We use the cultural algorithm to help us determine what the mix of strategies might have been like in the prehistoric population. The simulation results indeed suggest a mixed strategy consisting of defectors, inspectors, and trustors was necessary to produce results compatible with the archaeological data. It is suggested that the presence of defectors derives from the unreliability of the signal which increases under drought conditions and produced increased stress on Anasazi communities and may have contributed to their departure. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Coevolutionary free lunches

    Page(s): 721 - 735
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (496 KB) |  | HTML iconHTML  

    Recent work on the foundational underpinnings of black-box optimization has begun to uncover a rich mathematical structure. In particular, it is now known that an inner product between the optimization algorithm and the distribution of optimization problems likely to be encountered fixes the distribution over likely performances in running that algorithm. One ramification of this is the "No Free Lunch" (NFL) theorems, which state that any two algorithms are equivalent when their performance is averaged across all possible problems. This highlights the need for exploiting problem-specific knowledge to achieve better than random performance. In this paper, we present a general framework covering most optimization scenarios. In addition to the optimization scenarios addressed in the NFL results, this framework covers multiarmed bandit problems and evolution of multiple coevolving players. As a particular instance of the latter, it covers "self-play" problems. In these problems, the set of players work together to produce a champion, who then engages one or more antagonists in a subsequent multiplayer game. In contrast to the traditional optimization case where the NFL results hold, we show that in self-play there are free lunches: in coevolution some algorithms have better performance than other algorithms, averaged across all possible problems. However, in the typical coevolutionary scenarios encountered in biology, where there is no champion, the NFL theorems still hold. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Acknowledgment to Reviewers

    Page(s): 736 - 738
    Save to Project icon | Request Permissions | PDF file iconPDF (28 KB)  
    Freely Available from IEEE
  • 2005 Index

    Page(s): 739 - 744
    Save to Project icon | Request Permissions | PDF file iconPDF (114 KB)  
    Freely Available from IEEE
  • IEEE Computational Intelligence Society Information

    Page(s): c3
    Save to Project icon | Request Permissions | PDF file iconPDF (34 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Evolutionary Computation Information for authors

    Page(s): c4
    Save to Project icon | Request Permissions | PDF file iconPDF (31 KB)  
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Evolutionary Computation publishes archival quality original papers in evolutionary computation and related areas including nature-inspired algorithms, population-based methods, and optimization where selection and variation are integral, and hybrid systems where these paradigms are combined. Purely theoretical papers are considered as are application papers that provide general insights into these areas of computation.
 

Full Aims & Scope

Meet Our Editors

Editor-in-Chief

Dr. Kay Chen Tan (IEEE Fellow)

Department of Electrical and Computer Engineering

National University of Singapore

Singapore 117583

Email: eletankc@nus.edu.sg

Website: http://vlab.ee.nus.edu.sg/~kctan