By Topic

A Sampling-Based Approach to Reducing the Complexity of Continuous State Space POMDPs by Decomposition Into Coupled Perceptual and Decision Processes

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Rassool Fakoor ; Comput. Sci. & Eng. Dept., Univ. of Texas at Arlington, Arlington, TX, USA ; Manfred Huber

In this paper, we propose a method to reduce the complexity of solving POMDPs in continuous state spaces by decomposing them into separate, coupled perceptual and decision processes which leads to a reduction of the state space size of the decision learning problem. In our method, we reduce the state space of the POMDP by handling some aspects of the state space outside of the decision POMDP. To achieve this, the whole problem state space is decomposed into separate state spaces for the decision and perceptual process. The Perceptual process just serves to estimate aspects of the belief state while the decision process estimates the remainder and determines a policy. As a result, the decision process is modeled as a reduced state space POMDP. To allow the application of this method to continuous state spaces, the decision and the perceptual processes are here both handled by a sampling method within which this separation makes it possible to represent the POMDP with a smaller state space which leads to smaller sample sets for the decision POMDP and as a result to reduced representational and decision learning complexity. The goal here is to focus decision learning on the aspects of the space that are important for decision making while the observations and attributes that are important for estimating the state of the decision process are handled separately by the perceptual process. In this way, the separation into different processes can significantly reduce the complexity of decision learning. In the proposed framework and algorithm, Monte Carlo based sampling methods and corresponding sample set representations are used for both the perceptual and decision processes to be able to deal efficiently with continuous domains. We show analytically and experimentally how much the complexity of solving a POMDP can be reduced to increase the range of decision learning tasks that can be addressed.

Published in:

Machine Learning and Applications (ICMLA), 2012 11th International Conference on  (Volume:1 )

Date of Conference:

12-15 Dec. 2012