Skip to main content
SpringerLink
Log in

Computing a Partition Function of a Generalized Pattern-Based Energy over a Semiring

  • Published:
Theory of Computing Systems Aims and scope Submit manuscript

Abstract

Valued constraint satisfaction problems with ordered variables (VCSPO) are a special case of Valued CSPs in which variables are totally ordered and soft constraints are imposed on tuples of variables that do not violate the order. We study a restriction of VCSPO, in which soft constraints are imposed on a segment of adjacent variables and a constraint language \(\varvec{\Gamma }\) consists of \({\textbf {\{0,1\}}}\)-valued characteristic functions of predicates. This kind of potentials generalizes the so-called pattern-based potentials, which were applied in many tasks of structured prediction. For a constraint language \(\varvec{\Gamma }\) we introduce a closure operator, \( \overline{\varvec{\Gamma }^{\cap }}\supseteq \varvec{\Gamma }\), and give examples of constraint languages for which \(|\overline{\varvec{\Gamma }^{\cap }}|\) is small. If all predicates in \(\varvec{\Gamma }\) are cartesian products, we show that the minimization of a generalized pattern-based potential (or, the computation of its partition function) can be made in \({\varvec{\mathcal O}}(|\varvec{V}|\cdot |\varvec{D}|\varvec{^2} \cdot |\overline{\varvec{\Gamma }^{\cap }}|\varvec{^2})\) time, where \(\varvec{V}\) is a set of variables, \(\varvec{D}\) is a domain set. If, additionally, only non-positive weights of constraints are allowed, the complexity of the minimization task drops to \({\varvec{\mathcal O}}(|\varvec{V}|\cdot |\overline{\varvec{\Gamma }^{\cap }}| \cdot |\varvec{D}| \cdot \varvec{\max }_{\varrho \in \varvec{\Gamma }}\Vert \varrho \Vert \varvec{^2})\) where \(\Vert \varrho \Vert \) is the arity of \(\varrho \in \varvec{\Gamma }\). For a general language \(\varvec{\Gamma }\) and non-positive weights, the minimization task can be carried out in \({\varvec{\mathcal O}}(|\varvec{V}|\cdot |\overline{\varvec{\Gamma }^{\cap }}|\varvec{^2})\) time. We argue that in many natural cases \(\overline{\varvec{\Gamma }^{\cap }}\) is of moderate size, though in the worst case \(|\overline{\varvec{\Gamma }^{\cap }}|\) can blow up and depend exponentially on \(\varvec{\max }_{\varrho \in \varvec{\Gamma }}\Vert \varrho \Vert \).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. In some languages, such as agglutinative languages, the number of roots is substantially smaller than the number of possible words (due to a composite structure of words).

References

  1. Khanna, S., Sudan, M., Trevisan, L.: Constraint satisfaction: the approximability of minimization problems. In: Proceedings of Computational Complexity. Twelfth Annual IEEE Conference, pp. 282–296 (1997). https://doi.org/10.1109/CCC.1997.612323

  2. Cooper, M.C.: Reduction operations in fuzzy or valued constraint satisfaction. Fuzzy Sets and Systems 134(3), 311–342 (2003). https://doi.org/10.1016/S0165-0114(02)00134-3

  3. Cooper, M., de Givry, S., Sanchez, M., Schiex, T., Zytnicki, M.: Virtual arc consistency for weighted CSP. In: Twenty-third AAAI Conference on Artificial Intelligence. Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, p. 6. AAAI - Association for the Advancement of Artificial Intelligence, Chicago, United States (2008). https://hal.inrae.fr/hal-02752851

  4. Hell, P., Nešetřil, J.: On the complexity of h-coloring. Journal of Combinatorial Theory, Series B 48(1), 92–110 (1990). https://doi.org/10.1016/0095-8956(90)90132-J

  5. Jeavons, P.: On the algebraic structure of combinatorial problems. Theoretical Computer Science 200(1), 185–204 (1998). https://doi.org/10.1016/S0304-3975(97)00230-2

  6. Bulatov, A., Jeavons, P., Krokhin, A.: Classifying the complexity of constraints using finite algebras. SIAM J. Comput. 34(3), 720–742 (2005). https://doi.org/10.1137/S0097539700376676

  7. Bulatov, A.A.: A dichotomy theorem for nonuniform csps. In: 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pp. 319–330 (2017). https://doi.org/10.1109/FOCS.2017.37

  8. Feder, T., Vardi, M.Y.: Monotone monadic snp and constraint satisfaction. In: Proceedings of the Twenty-Fifth Annual ACM Symposium on Theory of Computing. STOC ’93, pp. 612–622. Association for Computing Machinery, New York, NY, USA (1993). 10.1145/167088.167245. https://doi.org/10.1145/167088.167245

  9. Zhuk, D.: A proof of the csp dichotomy conjecture. J. ACM 67(5) (2020). https://doi.org/10.1145/3402029

  10. Dechter, R., Pearl, J.: Tree clustering for constraint networks (research note). Artif. Intell. 38(3), 353–366 (1989). https://doi.org/10.1016/0004-3702(89)90037-4

  11. Gottlob, G., Leone, N., Scarcello, F.: Hypertree decompositions and tractable queries. Journal of Computer and System Sciences 64(3), 579–627 (2002). https://doi.org/10.1006/jcss.2001.1809

  12. Grohe, M., Marx, D.: Constraint solving via fractional edge covers. ACM Trans. Algorithms 11(1) (2014). https://doi.org/10.1145/2636918

  13. Cooper, M.C., Živný, S.: A new hybrid tractable class of soft constraint problems. In: Cohen, D. (ed.) Principles and Practice of Constraint Programming – CP 2010, pp. 152–166. Springer, Berlin, Heidelberg (2010)

  14. Kolmogorov, V., Rolínek, M., Takhanov, R.: Effectiveness of structural restrictions for hybrid csps. In: Elbassioni, K., Makino, K. (eds.) Algorithms and Computation - 26th International Symposium, ISAAC 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 566–577. Springer, Germany (2015). 10.1007/978-3-662-48971-0_48. 26th International Symposium on Algorithms and Computation, ISAAC 2015 ; Conference date: 09-12-2015 Through 11-12-2015

  15. Takhanov, R.: Hybrid vcsps with crisp and valued conservative templates. In: Tokuyama, T., Okamoto, Y. (eds.) 28th International Symposium on Algorithms and Computation, ISAAC 2017, vol. 92. Schloss Dagstuhl- Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing, Germany (2017). 10.4230/LIPIcs.ISAAC.2017.65. 28th International Symposium on Algorithms and Computation, ISAAC 2017 ; Conference date: 09-12-2017 Through 22-12-2017

  16. Takhanov, R.: Searching for an algebra on CSP solutions (2017)

  17. Ye, N., Lee, W.S., Chieu, H.L., Wu, D.: Conditional random fields with high-order features for sequence labeling. In: NIPS (2009)

  18. Takhanov, R., Kolmogorov, V.: Inference Algorithms for Pattern-based CRFs on Sequence Data, pp. 1182–1190 (2013). 30th International Conference on Machine Learning, ICML 2013 ; Conference date: 16-06-2013 Through 21-06-2013

  19. Qian, X., Jiang, X., Zhang, Q., Huang, X., Wu, L.: Sparse higher order conditional random fields for improved sequence labeling. In: ICML (2009)

  20. Takhanov, R., Assylbekov, Z.: Patterns versus characters in subword-aware neural language modeling. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.-S.M. (eds.) Neural Information Processing, pp. 157–166. Springer, Cham (2017)

  21. Takhanov, R., Kolmogorov, V.: Combining pattern-based crfs and weighted context-free grammars 26(1), 257–272 (2022). https://doi.org/10.3233/IDA-205623

  22. Impagliazzo, R., Paturi, R.: Complexity of k-sat. In: Proceedings of Fourteenth Annual IEEE Conference on Computational Complexity, pp. 237–240 (1999). https://doi.org/10.1109/CCC.1999.766282

  23. Bistarelli, S., Montanari, U., Rossi, F., Schiex, T., Verfaillie, G., Fargier, H.: Semiring-based csps and valued csps: Frameworks, properties, and comparison. Constraints 4(3), 199–240 (1999). https://doi.org/10.1023/A:1026441215081

  24. Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning. ICML ’01, pp. 282–289. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (2001)

  25. Istrail, S.: Statistical mechanics, three-dimensionality and np-completeness: I. universality of intracatability for the partition function of the ising model across non-planar surfaces (extended abstract). In: Proceedings of the Thirty-Second Annual ACM Symposium on Theory of Computing. STOC ’00, pp. 87–96. Association for Computing Machinery, New York, NY, USA (2000). https://doi.org/10.1145/335305.335316

  26. Sarawagi, S., Cohen, W.W.: Semi-markov conditional random fields for information extraction. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17, pp. 1185–1192. MIT Press, Cambridge, Massachusetts, USA (2005). https://proceedings.neurips.cc/paper/2004/file/eb06b9db06012a7a4179b8f3cb5384d3-Paper.pdf

  27. Rother, C., Kohli, P., Wei Feng, Jiaya Jia: Minimizing sparse higher order energy functions of discrete variables. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1382–1389 (2009). https://doi.org/10.1109/CVPR.2009.5206739

  28. Qian, X., Jiang, X., Zhang, Q., Huang, X., Wu, L.: Sparse higher order conditional random fields for improved sequence labeling. In: Proceedings of the 26th Annual International Conference on Machine Learning. ICML ’09, pp. 849–856. Association for Computing Machinery, New York, NY, USA (2009). 10.1145/1553374.1553483. https://doi.org/10.1145/1553374.1553483

  29. Komodakis, N., Paragios, N.: Beyond pairwise energies: Efficient optimization for higher-order mrfs. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2985–2992 (2009). https://doi.org/10.1109/CVPR.2009.5206846

  30. Cuong, N.V., Ye, N., Lee, W.S., Chieu, H.L.: Conditional random field with high-order dependencies for sequence labeling and segmentation. Journal of Machine Learning Research 15(28), 981–1009 (2014)

  31. Vieira, T., Cotterell, R., Eisner, J.: Speed-accuracy tradeoffs in tagging with variable-order CRFs and structured sparsity. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1973–1978. Association for Computational Linguistics, Austin, Texas (2016). 10.18653/v1/D16-1206. https://www.aclweb.org/anthology/D16-1206

  32. Lavergne, T., Yvon, F.: Learning the structure of variable-order CRFs: a finite-state perspective. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 433–439. Association for Computational Linguistics, Copenhagen, Denmark (2017). 10.18653/v1/D17-1044. https://www.aclweb.org/anthology/D17-1044

  33. Martins, A., Smith, N., Figueiredo, M., Aguiar, P.: Structured sparsity in structured prediction. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 1500–1511. Association for Computational Linguistics, Edinburgh, Scotland, UK. (2011). https://www.aclweb.org/anthology/D11-1139

  34. Mukanov, Z., Takhanov, R.: Learning the pattern-based crf for prediction of a protein local structure. Informatica 46(6), 135–141 (2022). https://doi.org/10.31449/inf.v46i6.3787

  35. Felzenszwalb, P., Oberlin, J.G.: Multiscale fields of patterns. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc., New York, USA (2014). https://proceedings.neurips.cc/paper/2014/file/2a38a4a9316c49e5a833517c45d31070-Paper.pdf

  36. Terrioux, C., Jegou, P.: Bounded backtracking for the valued constraint satisfaction problems. In: Rossi, F. (ed.) Principles and Practice of Constraint Programming – CP 2003, pp. 709–723. Springer, Berlin, Heidelberg (2003)

  37. de Givry, S., Schiex, T., Verfaillie, G.: Exploiting tree decomposition and soft local consistency in weighted csp. AAAI’06, pp. 22–27. AAAI Press, Boston, Massachusetts (2006)

  38. Ndiaye, S.N., Jégou, P., Terrioux, C.: Extending to soft and preference constraints a framework for solving efficiently structured problems. In: 2008 20th IEEE International Conference on Tools with Artificial Intelligence, vol. 1, pp. 299–306 (2008). https://doi.org/10.1109/ICTAI.2008.109

  39. Gottlob, G., Greco, G., Scarcello, F.: Tractable optimization problems through hypergraph-based structural restrictions. In: Albers, S., Marchetti-Spaccamela, A., Matias, Y., Nikoletseas, S., Thomas, W. (eds.) Automata, Languages and Programming, pp. 16–30. Springer, Berlin, Heidelberg (2009)

  40. Färnqvist, T.: Exploiting structure in CSP-related problems. PhD thesis (2013)

Download references

Acknowledgements

This research has been funded by Nazarbayev University under Faculty-development competitive research grants program for 2023-2025 Grant #20122022FD4131, PI Zh. Assylbekov. We would like to thank Anuar Sharafudinov for his help with writing a Java code that computes the closure of a simple language.

Author information

Authors and Affiliations

  1. Mathematics Department, School of Sciences and Humanities, Nazarbayev University, 53 Kabanbay Batyr Ave, Astana, 000010, Kazakhstan

    Rustem Takhanov

Authors
  1. Rustem Takhanov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rustem Takhanov.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Takhanov, R. Computing a Partition Function of a Generalized Pattern-Based Energy over a Semiring. Theory Comput Syst 67, 760–784 (2023). https://doi.org/10.1007/s00224-023-10128-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00224-023-10128-w

Keywords

Search

Navigation