Shlomi Maliah
Skip Abstract Section
Abstract
In contingent planning under partial observability with sensing actions, agents actively use sensing to discover meaningful facts about the world. Recent successful approaches translate the partially observable contingent problem into a non-deterministic fully observable problem, and then use a planner for non-deterministic planning. However, the translation may become very large, encumbering the task of the non-deterministic planner. We suggest a different approach—using an online contingent solver repeatedly to construct a plan tree. We execute the plan returned by the online solver until the next observation action, and then branch on the possible observed values, and replan for every branch independently. In many cases a plan tree can have an exponential width in the number of state variables, but the tree may have a structure that allows us to compactly represent it using a directed graph. We suggest a mechanism for tailoring such a graph that reduces both the computational effort and the storage space. Our method also handles non-deterministic domains, by identifying cycles in the plans. We present a set of experiments, showing our approach to scale better than state-of-the-art offline planners.
- [1] . 2009. A translation-based approach to contingent planning. In Proceedings of the IJCAI. 1623–1628. Google Scholar
Digital Library
- [2] . 2001. MBP: A model based planner. In Proceedings of the IJCAI’01 Workshop on Planning under Uncertainty and Incomplete Information. Seattle.Google Scholar
- [3] . 1997. Fast planning through planning graph analysis. Artificial Intelligence 90, 1–2 (1997), 281–300. Google ScholarDigital Library
- [4] . 2001. GPT: A tool for planning with uncertainty and partial information. In Proceedings of the IJCAI-01 Workshop on Planning with Uncertainty and Partial Information. Citeseer, 82–87.Google Scholar
- [5] . 2011. Planning under partial observability by classical replanning: Theory and experiments. In IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16–22, 2011. 1936–1941. Google ScholarDigital Library
- [6] . 2014. Belief tracking for planning with sensing: Width, complexity and approximations. Journal of Artificial Intelligence Research 50 (2014), 923–970.
DOI: https://doi.org/10.1613/jair.4475 Google ScholarDigital Library - [7] . 2010. Automatic derivation of finite-state machines for behavior control. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 24. Google ScholarDigital Library
- [8] . 2012. A multi-path compilation approach to contingent planning. In Proceedings of the 26th AAAI Conference on Artificial Intelligence. Google ScholarDigital Library
- [9] . 2012. Replanning in domains with partial information and sensing actions. Journal of Artificial Intelligence Research 45, 1 (2012), 565–600. Google ScholarDigital Library
- [10] . 2014. On the properties of belief tracking for online contingent planning using regression. In Proceedings of the ECAI 2014-21st European Conference on Artificial Intelligence. 147–152. Google ScholarDigital Library
- [11] . 2016. Online belief tracking using regression for contingent planning. Artificial Intelligence 241 (2016), 131–152.
DOI: https://doi.org/10.1016/j.artint.2016.08.005 Google ScholarDigital Library - [12] . 2006. Planning graph heuristics for belief space search. Journal of AI Research 26, 1 (2006), 35–99. Google ScholarDigital Library
- [13] . 1998. A survey of POMDP applications. In Proceedings of the Working Notes of AAAI 1998 Fall Symposium on Planning with Partially Observable Markov Decision Processes, Vol. 1724.Google Scholar
- [14] . 2013. Incremental pruning: A simple, fast, exact method for partially observable markov decision processes. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI’97). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 54–61. Google ScholarDigital Library
- [15] . 2000. Conformant planning via symbolic model checking. Journal of Artificial Intelligence Research 13 (2000), 305–338. Google ScholarDigital Library
- [16] . 2015. On the progression of knowledge and belief for nondeterministic actions in the situation calculus. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. Google ScholarDigital Library
- [17] . 2006. Plan Stability: Replanning versus Plan Repair. In Proceedings of the ICAPS, Vol. 6. 212–221. Google ScholarDigital Library
- [18] . 2008. Sparse stochastic finite-state controllers for POMDPs. In Proceedings of the UAI. 256–263. Google ScholarDigital Library
- [19] . 2001. The FF planning system: Fast plan generation through heuristic search. JAIR 14 (2001), 253–302. Google ScholarDigital Library
- [20] . 2005. Contingent planning via heuristic forward search with implicit belief states. In Proceedings of the ICAPS, Vol. 2005. Google ScholarDigital Library
- [21] . 2011. A generic framework and solver for synthesizing finite-state controllers. In Proceedings of the AAAI 2011 Workshop on Generalized Planning.Google Scholar
- [22] . 2018. Finite-state controllers of POMDPs using parameter synthesis. In Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, August 6-10, 2018, and (Eds.), AUAI Press, 519–529.Google Scholar
- [23] . 2018. Specifying plausibility levels for iterated belief change in the situation calculus. In Proceedings of the 16th International Conference on Principles of Knowledge Representation and Reasoning.Google Scholar
- [24] . 2016. Computing contingent plans using online replanning. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona.3159–3165. Google ScholarDigital Library
- [25] . 2008. Sarsop: Efficient point-based pomdp planning by approximating optimally reachable belief spaces. In Proceedings of the Robotics: Science and Systems, Vol. 2008. Citeseer.Google ScholarCross Ref
- [26] . 2014. Partially observable online contingent planning using landmark heuristics. In Proceedings of the 24th International Conference on Automated Planning and Scheduling, ICAPS. Google ScholarDigital Library
- [27] . 2012. Improved non-deterministic planning by exploiting state relevance. In Proceedings of the 22nd International Conference on Automated Planning and Scheduling, ICAPS. Google ScholarDigital Library
- [28] . 2014. Computing contingent plans via fully observable non-deterministic planning. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Google ScholarDigital Library
- [29] . 2014. Compiling contingent planning into classical planning: New translations and results. In Proceedings of the ICAPS Workshop on Models and Paradigms for Planning under Uncertainty.Google Scholar
- [30] . 2009. Compiling uncertainty away in conformant planning problems with bounded width. Journal of Artificial Intelligence Research 35 (2009), 623–675. Google ScholarDigital Library
- [31] . 2003. Bounded finite state controllers. Advances in Neural Information Processing Systems 16 (2003), 823–830. Google ScholarDigital Library
- [32] . 2008. Regression for classical and nondeterministic planning. In Proceedings of the ECAI 2008-18th European Conference on Artificial Intelligence, Patras, Greece, July 21–25, 2008, Proceedings. 568–572.
DOI: https://doi.org/10.3233/978-1-58603-891-5-568 Google ScholarDigital Library - [33] . 2009. Planning and SAT. Handbook of Satisfiability 185 (2009), 483–504.Google Scholar
- [34] . 2010. Symbolic dynamic programming for first-order POMDPs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 24. Google ScholarDigital Library
- [35] . 2011. Replanning in domains with partial information and sensing actions. In Proceedings of the IJCAI. 2021–2026. Google ScholarDigital Library
- [36] . 2008. Efficient ADD operations for point-based algorithms. In Proceedings of the ICAPS. 330–337. Google ScholarDigital Library
- [37] . 2013. A survey of point-based POMDP solvers. Autonomous Agents and Multi-Agent Systems 27, 1 (2013), 1–51. Google ScholarDigital Library
- [38] . 2019. Comparative criteria for partially observable contingent planning. Autonomous Agents and Multi-Agent Systems 33, 5 (2019), 481–517. Google ScholarDigital Library
- [39] . 2008. Symbolic heuristic search value iteration for factored POMDPs. In Proceedings of the AAAI. 1088–1093. Google ScholarDigital Library
- [40] . 1978. The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs. Operations Research 26, 2 (1978), 282–304. Google ScholarDigital Library
- [41] . 2011. On the impact of belief state representation in planning under uncertainty. In Proceedings of the IJCAI. 2856–2857. Google ScholarDigital Library
- [42] . 2011. On the effectiveness of CNF and DNF representations in contingent planning. In Proceedings of the IJCAI. 2033–2038. Google ScholarDigital Library
- [43] . 2011. On the effectiveness of belief state representation in contingent planning. In Proceedings of the AAAI. Google ScholarDigital Library
- [44] . 2011. Conjunctive representations in contingent planning: Prime implicates versus minimal CNF formula. In Proceedings of the AAAI. Google ScholarDigital Library
- [45] . 2007. FF-replan: A baseline for probabilistic planning. In Proceedings of the ICAPS, Vol. 7. 352–359. Google ScholarDigital Library
- [46] . 2020. Computing superior counter-examples for conformant planning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 10017–10024.Google ScholarCross Ref
Index Terms
- Computing Contingent Plan Graphs using Online Planning
Recommendations
Landmark-based heuristic online contingent planning
In contingent planning problems, agents have partial information about their state and use sensing actions to learn the value of some variables. When sensing and actuation are separated, plans for such problems can often be viewed as a tree of sensing ...
Comparative Criteria for Partially Observable Contingent Planning
AAMAS '19: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent SystemsIn contingent planning under partial observability with sensing actions, the solution can be represented as a plan tree, branching on various possible observations. Typically, one seeks a satisfying plan leading to a goal state at each leaf. In many ...
Unavoidable deadends in deterministic partially observable contingent planning
AbstractTraditionally, a contingent plan, branching on the observations an agent obtains throughout plan execution, must reach a goal state from every possible initial state. However, in many real world problems, no such plan exists. Yet, there are plans ...
Comments