State-Continuity Approximation of Markov Decision Processes via Finite Element Methods for Autonomous System Planning

Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this letter, we propose a novel solution to this decision-theoretic planning problem that directly obtains the continuous value function with only the first and second moments of the transition probabilities, alleviating the requirement for an explicit transition model in the literature. We achieve this by expressing the value function as a linear combination of basis functions and approximating the Bellman equation by a partial differential equation, where the value function can be naturally constructed using a finite element method. We have validated our approach via extensive simulations, and the evaluations reveal that compared to baseline methods, our solution leads to better planning results in terms of path smoothness, travel distance, and time costs.