Counter-Strike Deathmatch with Large-Scale Behavioural Cloning

This paper describes an AI agent that plays the modern first-person-shooter (FPS) video game ‘Counter-Strike; Global Offensive’ (CSGO) from pixel input. The agent, a deep neural network, matches the performance of a casual human gamer on the deathmatch game mode whilst adopting a humanlike play style. Much previous research has focused on games with convenient APIs and low-resolution graphics, allowing them to be run cheaply at scale. This is not the case for CSGO, with system requirements orders of magnitude higher than previously studied FPS games. This limits the quantity of on-policy data that can be generated, precluding pure reward-driven reinforcement learning (RL) algorithms. Our solution uses a two-stage behavioural cloning methodology; 1) Pre-train on a large dataset scraped from human play on public servers (5.5 million frames or 95 hours) where actions are labelled in an automated way. 2) Fine-tune on a small dataset of clean expert demonstrations (190 thousand frames or 3 hours). This scale is an order of magnitude larger than prior work on imitation learning in FPS games, whilst being far more data efficient than pure RL algorithms. Video introduction: https://youtu.be/rnz3lmfSHv0 Code, model & datasets: https://github.com/TeaPearce