By Topic

Learning to Drive in the Open Racing Car Simulator Using Online Neuroevolution

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Cardamone, L. ; Dipt. di Elettron. e Inf., Politec. di Milano, Milan, Italy ; Loiacono, D. ; Lanzi, P.L.

In this paper, we applied online neuroevolution to evolve nonplayer characters for The Open Racing Car Simulator (TORCS). While previous approaches allowed online learning with performance improvements during each generation, our approach enables a finer grained online learning with performance improvements within each lap. We tested our approach on three tracks using two methods of online neuroevolution (NEAT and rtNEAT) combined with four evaluation strategies ( -greedy, -greedy-improved, softmax, and interval-based) taken from the literature. We compared the eight resulting configurations on several driving tasks involving the learning of a driving behavior for a specific track, its adaptation to a new track, and the generalization capability to unknown tracks. The results we present show that, notwithstanding the several challenges that online learning poses, our approach 1) can successfully evolve drivers from scratch, 2) can also be used to transfer evolved knowledge to other tracks, and 3) can generalize effectively producing controllers that can drive on difficult unseen tracks. Our results also suggest that the approach performs better when coupled with online NEAT and also indicate that -greedy-improved and softmax are generally better than the other evaluation strategies. A comparison with typical offline neuroevolution suggests that online neuroevolution can be competitive and even outperform traditional offline approaches on more difficult tracks while providing all the interesting features of online learning. Overall, we believe that this study may represent an initial step toward the application of online neuroevolution in games.

Published in:

Computational Intelligence and AI in Games, IEEE Transactions on  (Volume:2 ,  Issue: 3 )