Expandable-Partially Observable Markov Decision-Process Framework for Modeling and Analysis of Autonomous Vehicle Behavior

When modeling decision-making in autonomous vehicles (AVs), only a fraction of the required information is usually available at the start due to the presence of uncertainty in the knowledge of the state of the system and the environment. This article presents a probabilistic modeling framework based on a combination of partially observable Markov decision-process (POMDP) models and heuristics. In this article, POMDPs are expanded and redefined as expandable-POMDPs to account for uncertainty and unexplainable observations. Specifically, this approach incorporates new hidden states in the model and expands associated probability distributions in the model to account for partial knowledge and unexplainable observations. An online policy estimation algorithm is presented. This algorithm employs heuristic search combined with distance-based clustering to estimate long-term rewards in a finite-horizon look-ahead scheme for expandable-POMDPs. Specifically, heuristics are employed in model design and online policy evaluation to circumvent state-space explosion and reduce high computational complexity. Finally, the expandable-POMDP models are employed for designing multiple context-specific models for tactical planning and decision-making by AVs in simulated scenarios characterized by partial observability and uncertainty. One (or more) expandable-POMDP model is invoked when the contextual pattern of the ongoing scenario fits the objective of the tactical planning model.