Scenario Generation-Based Training in Simulation: Pilot Study

Scenario generation-based training in simulated environments has recently gained importance, since real life training environments can be costly, risky, time consuming, and requires substantial resources. In this paper, we propose a narrative-based scenario generation methodology for training, measuring, and analysing the trainee performance level in simulation. We use a driving simulator as an application domain for the training process. Furthermore, we utilised an autonomous Artificial Intelligence (AI) agent for the training experiments. Using the AI agent for practising the generated scenarios offers benefits in many aspects. Firstly, the AI agent simulates the behaviour of a human (trainee). Secondly, it is easy to collect the performance data with the AI agent, as compared with recruiting many trainees for data collection, particularly for the early stage of validation and verification of the proposed scenario generation methodology. We formulate an experimental study to measure and assess the AI agent's behaviours in scenarios with different levels of complexity. The collected performance metrics are used to evaluate the efficiency of the designed scenarios and its capabilities in capturing the variations in performance levels. The empirical results depict that the AI agent's behaviours pertain to the level of scenario complexity including varying weather conditions.