Decision Making Under Uncertainty: Theory and Application [Bookshelf]

The unmanned air vehicles and self-driving cars of the future will require a high degree of autonomy. Not only will they need feedback loops that are conducive to a wide variety of environmental conditions, but they will also require higher levels of reasoning and planning that can efficiently handle unexpected conditions and scenarios. Since the dynamics of such systems will have inherent uncertainty (and the available environmental sensors like radar, lidar, and computer vision have high levels of noise and associated uncertainty), solutions will require statistical modeling and computation. To this end, this text discusses a framework for formulating these types of problems as partially observable Markov decision processes (POMDPs). The book proposes solving POMDPs by using approximate dynamic programming. The content of the book is divided into two main parts. Part I outlines the basic theory, and Part II contains several chapters that detail nontrivial applications of the theory to solve several problems that are currently of interest. Kochenderfer is the primary author of Part I, but each chapter in Part II is written by a different set of authors from different subject domains, providing a compelling argument that POMDPs are a suitable modeling and solution technique for a variety of different applications. This book is a timely and well-written introduction to modeling complex decision and control problems using the POMDP framework. The author has done an admirable job of bringing together various approximation techniques that have been developed to make the solution of this class of problems tractable. Since the techniques introduced in the book can be applied to a wide variety of problems that include self-driving and self-flying vehicles, the book is a welcome addition to the literature.