Skip to main content
SpringerLink
Log in

Model Checking for Probabilistic Multiagent Systems

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In multiagent systems, agents usually do not have complete information of the whole system, which makes the analysis of such systems hard. The incompleteness of information is normally modelled by means of accessibility relations, and the schedulers consistent with such relations are called uniform. In this paper, we consider probabilistic multiagent systems with accessibility relations and focus on the model checking problem with respect to the probabilistic epistemic temporal logic, which can specify both temporal and epistemic properties. However, the problem is undecidable in general. We show that it becomes decidable when restricted to memoryless uniform schedulers. Then, we present two algorithms for this case: one reduces the model checking problem into a mixed integer non-linear programming (MINLP) problem, which can then be solved by Satisfiability Modulo Theories (SMT) solvers, and the other is an approximate algorithm based on the upper confidence bounds applied to trees (UCT) algorithm, which can return a result whenever queried. These algorithms have been implemented in an existing model checker and then validated on experiments. The experimental results show the efficiency and extendability of these algorithms, and the algorithm based on UCT outperforms the one based on MINLP in most cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Seuken S, Zilberstein S. Formal models and algorithms for decentralized decision making under uncertainty. Autonomous Agents and Multi-Agent Systems, 2008, 17(2): 190–250. DOI: https://doi.org/10.1007/s10458-007-9026-5.

    Article  Google Scholar 

  2. Bernstein D S, Zilberstein S, Washington R, Bresina J L. Planetary rover control as a Markov decision process. In Proc. the 6th Int. Symp. Artificial Intelligence, Robotics, and Automation in Space, Jun. 2001.

  3. Nair R, Varakantham P, Tambe M, Yokoo M. Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs. In Proc. the 20th National Conf. Artificial Intelligence and the 17th Innovative Applications of Artificial Intelligence Conference, Jul. 2005, pp.133–139. DOI: 10.5555/1619332.1619356.

  4. Pynadath D V, Tambe M. The communicative multiagent team decision problem: Analyzing teamwork theories and models. Journal of Artificial Intelligence Research, 2002, 16: 389–423. DOI: https://doi.org/10.1613/jair.1024.

    Article  MathSciNet  MATH  Google Scholar 

  5. Kaźmierczak P, Ågotnes T, Jamroga W. Multi-agency is coordination and (limited) communication. In Proc. the 17th Int. Conf. Principles and Practice of Multi-Agent Systems, Dec. 2014, pp.91–106. DOI: https://doi.org/10.1007/978-3-319-13191-7_8.

  6. Delgado C, Benevides M. Verification of epistemic properties in probabilistic multi-agent systems. In Proc. the 7th German Conference on Multiagent System Technologies, Sept. 2009, pp.16–28. DOI: 10.1007/978-3-642-04143-3_3.

  7. Fagin R, Halpern J Y, Moses Y, Vardi M Y. Reasoning About Knowledge. The MIT Press, 2004. DOI: https://doi.org/10.7551/mitpress/5803.001.0001.

  8. Jonsson B, Larsen K G. Specification and refinement of probabilistic processes. In Proc. the 6th Annual IEEE Symposium on Logic in Computer Science, Jul. 1991, pp.266–277. DOI: 10.1109/LICS.1991.151651.

  9. Baier C, Katoen J P. Principles of Model Checking. The MIT Press, 2008.

  10. De Moura L, Bjørner N. Z3: An efficient SMT solver. In Proc. the 14th Int. Conf. Tools and Algorithms for the Construction and Analysis of Systems, Mar. 2008, pp.337–340. DOI: https://doi.org/10.1007/978-3-540-78800-3_24.

  11. Kocsis L, Szepesvári C. Bandit based Monte-Carlo planning. In Proc. the 17th European Conf. Machine Learning, Sept. 2006, pp.282–293. DOI: https://doi.org/10.1007/11871842_29.

  12. Keller T, Eyerich P. PROST: Probabilistic planning based on UCT. In Proc. the 22nd International Conference on Automated Planning and Scheduling, June 2012, pp.119–127. DOI: 10.1609/icaps.v22i1.13518.

  13. Coles A, Coles A, Olaya A G, Jiménez S, López C L, Sanner S, Yoon S. A survey of the seventh international planning competition. AI Magazine, 2012, 33(1): 83–88. DOI: https://doi.org/10.1609/aimag.v33i1.2392.

    Article  Google Scholar 

  14. Vallati M, Chrpa L, Grześ M, McCluskey T L, Roberts M, Sanner S. The 2014 international planning competition: Progress and trends. AI Magazine, 2015, 36(3): 90–98. DOI: https://doi.org/10.1609/aimag.v36i3.2571.

    Article  Google Scholar 

  15. Hartmanns A, Klauck M, Parker D, Quatmann T, Ruijters E. The quantitative verification benchmark set. In Proc. the 25th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, Apr. 2019, pp.344–350. DOI: 10.1007/978-3-030-17462-0_20.

  16. Fu C, Turrini A, Huang X W, Song L, Feng Y, Zhang L J. Model checking probabilistic epistemic logic for probabilistic multiagent systems. In Proc. the 27th International Joint Conference on Artificial Intelligence, Jul. 2018, pp.4757–4763. DOI: 10.24963/ijcai.2018/661.

  17. Huang X W, Luo C. A logic of probabilistic knowledge and strategy. In Proc. the 12th International Conference on Autonomous Agents and Multi-Agent Systems, May 2013, pp.845–852. DOI: 10.5555/2484920.2485055.

  18. Schobbens P Y. Alternating-time logic with imperfect recall. Electronic Notes in Theoretical Computer Science, 2004, 85(2): 82–93. DOI: https://doi.org/10.1016/S1571-0661(05)82604-0.

    Article  MathSciNet  MATH  Google Scholar 

  19. Jamroga W, Van Der Hoek W. Agents that know how to play. Fundamenta Informaticae, 2004, 63(2/3): 185–219.

    MathSciNet  MATH  Google Scholar 

  20. Rabin M O. Probabilistic automata. Information and Control, 1963, 6(3): 230–245. DOI: https://doi.org/10.1016/S0019-9958(63)90290-0.

    Article  MathSciNet  MATH  Google Scholar 

  21. Paz A. Introduction to Probabilistic Automata. Academic Press, 1971. DOI: https://doi.org/10.1016/C2013-0-11297-4.

  22. Madani O, Hanks S, Condon A. On the undecidability of probabilistic planning and related stochastic optimization problems. Artificial Intelligence, 2003, 147(1/2): 5–34. DOI: https://doi.org/10.1016/S0004-3702(02)00378-8.

    Article  MathSciNet  MATH  Google Scholar 

  23. Madani O, Hanks S, Condon A. On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems. In Proc. the 16th National Conference on Artificial Intelligence and the 11th Innovative Applications of Artificial Intelligence, Jul. 1999, pp.541–548. DOI: 10.5555/315149.315395.

  24. Mundhenk M. The complexity of planning with partiallyobservable Markov decision processes. Computer Science Technical Report TR2000-376, Dartmouth College, 2000. https://digitalcommons.dartmouth.edu/cs_tr/176/, Sept. 2023.

  25. Bellman R E. Dynamic Programming. Princeton University Press, 1957.

  26. Kolobov A, Mausam, Weld D. LRTDP versus UCT for online probabilistic planning. In Proc. the 26th AAAI Conference on Artificial Intelligence, Jul. 2012, pp.1786–1792. DOI: 10.1609/aaai.v26i1.8362.

  27. Younes H L S, Simmons R G. Probabilistic verification of discrete event systems using acceptance sampling. In Proc. the 14th Int. Conf. Computer Aided Verification, Jul. 2002, pp.223–235. DOI: https://doi.org/10.1007/3-540-45657-0_17.

  28. Hérault T, Lassaigne R, Magniette F, Peyronnet S. Approximate probabilistic model checking. In Proc. the 5th Int. Conf. Verification, Model Checking, and Abstract Interpretation, Jan. 2004, pp.73–84. DOI: 10.1007/978-3-540-24622-0_8.

  29. Henriques D, Martins J G, Zuliani P, Platzer A, Clarke E M. Statistical model checking for Markov decision processes. In Proc. the 9th International Conference on Quantitative Evaluation of Systems, Sept. 2012, pp.84–93. DOI: 10.1109/QEST.2012.19.

  30. D’Argenio P R, Hartmanns A, Sedwards S. Lightweight statistical model checking in nondeterministic continuous time. In Proc. the 8th Int. Symp. Leveraging Applications of Formal Methods, Verification and Validation, Nov. 2018, pp.336–353. DOI: https://doi.org/10.1007/978-3-030-03421-4_22.

  31. Ashok P, Kretínský J, Weininger M. PAC statistical model checking for Markov decision processes and stochastic games. In Proc. the 31st International Conference on Computer Aided Verification, Jul. 2019, pp.497–519. DOI: 10.1007/978-3-030-25540-4_29.

  32. Hahn E M, Li Y, Schewe S, Turrini A, Zhang L J. IscasMC: A Web-based probabilistic model checker. In Proc. the 19th Int. Symp. Formal Methods, May 2014, pp.312–317. DOI: https://doi.org/10.1007/978-3-319-06410-9_22.

  33. Kwiatkowska M, Norman G, Parker D, Santos G. PRISM-games 3.0: Stochastic game verification with concurrency, equilibria and time. In Proc. the 32nd International Conference on Computer Aided Verification, Jul. 2020, pp.475–487. DOI: 10.1007/978-3-030-53291-8_25.

  34. Kwiatkowska M, Parker D, Simaitis A. Strategic analysis of trust models for user-centric networks. In Proc. the 1st International Workshop on Strategic Reasoning, Mar. 2013, pp.53–59. DOI: 10.4204/EPTCS.112.10.

  35. Kwiatkowska M, Norman G, Parker D, Santos G. Automatic verification of concurrent stochastic systems. Formal Methods in System Design, 2021, 58(1/2): 188–250. DOI: https://doi.org/10.1007/s10703-020-00356-y.

    Article  MATH  Google Scholar 

  36. Kwiatkowska M, Norman G, Parker D, Santos G. Multiplayer equilibria verification for concurrent stochastic games. In Proc. the 17th International Conference on Quantitative Evaluation of Systems, Aug. 2020, pp.74–95. DOI: 10.1007/978-3-030-59854-9_7.

  37. Huang X W, Su K L, Zhang C Y. Probabilistic alternating-time temporal logic of incomplete information and synchronous perfect recall. In Proc. the 26th AAAI Conference on Artificial Intelligence, Jul. 2012, pp.765–771. DOI: 10.1609/aaai.v26i1.8214.

  38. Huang X W, Van Der Meyden R. An epistemic strategy logic. ACM Trans. Computational Logic, 2018, 19(4): Article No. 26. DOI: https://doi.org/10.1145/3233769.

  39. Chatterjee K, Doyen L, Henzinger T A. A survey of partial- observation stochastic parity games. Formal Methods in System Design, 2013, 43(2): 268–284. DOI: https://doi.org/10.1007/s10703-012-0164-2.

    Article  MATH  Google Scholar 

  40. Chatterjee K, Doyen L. Partial-observation stochastic games: How to win when belief fails. ACM Trans. Computational Logic, 2014, 15(2): Article No. 16. DOI: https://doi.org/10.1145/2579821.

  41. Schnoor H. Strategic planning for probabilistic games with incomplete information. In Proc. the 9th Int. Conf. Autonomous Agents and Multiagent Systems, May 2010, pp.1057–1064. DOI: https://doi.org/10.5555/1838206.1838349.

  42. Huang X W, Luo C, Van Der Meyden R. Symbolic model checking of probabilistic knowledge. In Proc. the 13th Conference on Theoretical Aspects of Rationality and Knowledge, Jul. 2011, pp.177–186. DOI: 10.1145/2000378.2000399.

  43. Wan W, Bentahar J, Hamza A B. Model checking epistemic-probabilistic logic using probabilistic interpreted systems. Knowledge-Based Systems, 2013, 50: 279–295. DOI: https://doi.org/10.1016/j.knosys.2013.06.017.

    Article  Google Scholar 

  44. Sultan K, Bentahar J, Wan W, Al-Saqqar F. Modeling and verifying probabilistic multi-agent systems using knowledge and social commitments. Expert Systems with Applications, 2014, 41(14): 6291–6304. DOI: https://doi.org/10.1016/j.eswa.2014.04.008.

    Article  Google Scholar 

  45. Kwiatkowska M Z, Norman G, Parker D. PRISM 4.0: Verification of probabilistic real-time systems. In Proc. the 23rd Int. Conf. Computer Aided Verification, Jul. 2011, pp.585–591. DOI: https://doi.org/10.1007/978-3-642-22110-1_47.

  46. Lomuscio A, Qu H Y, Raimondi F. MCMAS: An opensource model checker for the verification of multi-agent systems. International Journal on Software Tools for Technology Transfer, 2017, 19(1): 9–30. DOI: https://doi.org/10.1007/s10009-015-0378-x.

    Article  Google Scholar 

  47. Gammie P, Van Der Meyden R. MCK: Model checking the logic of knowledge. In Proc. the 16th International Conference on Computer Aided Verification, Jul. 2004, pp.479–483. DOI: 10.1007/978-3-540-27813-9_41.

  48. Kumar A, Zilberstein S. Dynamic programming approximations for partially observable stochastic games. In Proc. the 22nd International FLAIRS Conference, May 2009, pp.547–552.

  49. Winterer L, Wimmer R, Jansen N, Becker B. Strengthening deterministic policies for POMDPs. In Proc. the 12th International Symposium on NASA Formal Methods, May 2020, pp.115–132. DOI: 10.1007/978-3-030-55754-6_7.

  50. Oliehoek F A, Amato C. A Concise Introduction to Decentralized POMDPs. Springer, 2016. DOI: https://doi.org/10.1007/978-3-319-28929-8.

  51. Norman G, Parker D, Zou X Y. Verification and control of partially observable probabilistic systems. Real-Time Systems, 2017, 53(3): 354–402. DOI: https://doi.org/10.1007/s11241-017-9269-4.

    Article  MATH  Google Scholar 

  52. Pnueli A. The temporal logic of programs. In Proc. the 18th Annual Symp. Foundations of Computer Science, Oct. 31–Nov. 2, 1977, pp.46–57. DOI: 10.1109/SFCS.1977.32.

  53. Huang X W, Kwiatkowska M, Olejnik M. Reasoning about cognitive trust in stochastic multiagent systems. ACM Trans. Computational Logic, 2019, 20(4): Article No. 21. DOI: https://doi.org/10.1145/3329123.

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, 100190, China

    Chen Fu, Andrea Turrini & Li-Jun Zhang

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Chen Fu & Li-Jun Zhang

  3. Institute of Intelligent Software, Guangzhou, 511455, China

    Andrea Turrini & Li-Jun Zhang

  4. Department of Computer Science, University of Liverpool, Liverpool, L693BX, U.K.

    Xiaowei Huang

  5. Microsoft Research, Beijing, 100190, China

    Lei Song

  6. Centre for Quantum Software and Information, University of Technology Sydney, Sydney, 2007, Australia

    Yuan Feng

Authors
  1. Chen Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Turrini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaowei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Li-Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li-Jun Zhang.

Supplementary Information

ESM 1

(PDF 139 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, C., Turrini, A., Huang, X. et al. Model Checking for Probabilistic Multiagent Systems. J. Comput. Sci. Technol. 38, 1162–1186 (2023). https://doi.org/10.1007/s11390-022-1218-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-022-1218-6

Keywords

Search

Navigation