Determination of Initial Projectile Velocity in the Presence of Static Fields Using Deep Actor Critic Method

The movement of a projectile, which can be a mass or a charge, through a static field, can be important in various fields of physics ranging from particle physics to rocket science. In many cases, the movement of the projectile can be solely dependent on the initial velocity provided to the projectile. The determination of this initial velocity can be very complicated if the existing static field is too complex. In such cases, neural networks can be the key to modelling the environment and accurately estimate the initial velocity. This AI-driven velocity estimation can be extremely difficult using supervised learning due to the lack of seen examples of the state (initial position) to action (velocity) relation, which is also known as the policy. In this work, this problem is treated as a continuous-action-space contextual bandit problem and a neural network-based actor critic method is used to model the environment and determine the action policy. This method is tested on two different static field configurations using a Python-based GUI named Pygame. After training, the success rates of the projectile hitting the targeted destinations are 94.8% and 77.9% respectively, which implies that the proposed method can successfully model the action policy with high accuracy.