Differentially private multi-agent constraint optimization

Abstract

Distributed constraint optimization (DCOP) is a framework in which multiple agents with private constraints (or preferences) cooperate to achieve a common goal optimally. DCOPs are applicable in several multi-agent coordination/allocation problems, such as vehicle routing, radio frequency assignments, and distributed scheduling of meetings. However, optimization scenarios may involve multiple agents wanting to protect their preferences’ privacy. Researchers propose privacy-preserving algorithms for DCOPs that provide improved privacy protection through cryptographic primitives such as partial homomorphic encryption, secret-sharing, and secure multiparty computation. These privacy benefits come at the expense of high computational complexity. Moreover, such an approach does not constitute a rigorous privacy guarantee for optimization outcomes, as the result of the computation may compromise agents’ preferences. In this work, we show how to achieve privacy, specifically Differential Privacy, by randomizing the solving process. In particular, we present P-Gibbs, which adapts the current state-of-the-art algorithm for DCOPs, namely SD-Gibbs, to obtain differential privacy guarantees with much higher computational efficiency. Experiments on benchmark problems such as Ising, graph-coloring, and meeting-scheduling show P-Gibbs’ privacy and performance trade-off for varying privacy budgets and the SD-Gibbs algorithm. More concretely, we empirically show that P-Gibbs provides fair solutions for competitive privacy budgets.

Appendices

Appendix 1 Comparing P-Gibbs’ quality of solution with DPOP

Complete algorithms like DPOP fail to solve sufficiently large problems. More concretely, in our experiments, the pyDCOP’s DPOP solver timed out after 24 h for (i) an Ising benchmark instance with 10 variables/agents, (ii) a graph-coloring benchmark instance with 12 variables/agents and $|D|=8$, (iii) a meeting-scheduling benchmark instance with 25 variables/agents and $|D|=30$. Recall that in Sect. 6, we conduct experiments on significantly larger problems than these.

As private algorithms built atop DPOP (i.e., P-DPOP, P$^{3/2}$-DPOP and P$^2$-DPOP) use computationally expensive cryptographic primitives, these algorithms are less efficient than DPOP [3, 17]. Given this evidence, we conclude that P-Gibbs is significantly more scalable than the DPOP family of algorithms.

Fig. 6

The average and standard deviation of P-Gibbs’ Solution Quality (SQ) for different privacy budgets, compared to DPOP. Note that, the case with $\epsilon =0.046$ corresponds to P-Gibbs$_\infty$ as $\gamma =\infty$

Next, we want to compare the quality of solutions given by the DPOP family of algorithms and P-Gibbs. As each of P-DPOP, P$^{3/2}$-DPOP, and P$^2$-DPOP do not any noise/randomness to the computation process, it suffices to compare the solution quality of DPOP and SD-Gibbs (refer Definition 6). We begin by setting up the benchmark problems.

1.1 Solution Quality

Benchmarks. As in Sect. 6, the instances are created using pyDCOP’s generate option.⁷

• 1. Ising [23]. We generate 5 sample Ising problems with variables/agents between [5, 8] and $D_i=\{0,1\},\forall i$. The constraints are of two types: (i) binary constraints whose strength is sampled from $\mathcal {U}[\beta ,\beta ]$ where $\beta \in [1,10)$ and (ii) unary constraints whose strength is sampled from $\mathcal {U}[-\rho ,\rho ]$ where $\rho \in [0.05,0.9)$. Ising is a minimization problem.

• 2. Graph-coloring (GC). We generate 5 sample graph-coloring problems. The problems are such that the number of agents/variables lies between [8, 12] and agents’ domain size between [10, 20). Each constraint is a random integer taken from (0, 10). Graph-coloring is a minimization problem.

• 3. Meeting-scheduling (MS). We generate 5 sample meeting-scheduling problems. The problem instances are such that the number of agents and variables lies between [15, 20] with the number of slots, i.e., the domain for each agent randomly chosen from [20, 30]. Each constraint is a random integer taken from (0, 100), while each meeting may randomly occupy [1, 5] slots. Meeting-scheduling is a maximization problem.

Results. Fig. 6 depicts the results. As previously observed in Fig. 4, P-Gibbs’ solution quality improves as the problem size (m) increases. As DPOP does not scale, we see that the solution quality remains around 50–75%, with an increase in the quality with an increase in $\epsilon$. Furthermore, as P-DPOP, P$^{3/2}$-DPOP, and P$^2$-DPOP do not add noise/randomness in the solution process, we argue that these results, although hold for them.

Appendix 2 Comparing P-Gibbs’ quality of solution with max-sum

Fig. 7

The average and standard deviation of P-Gibbs’ Solution Quality (SQ) for different privacy budgets, compared to Max-Sum. Note that, the case with $\epsilon =0.046$ corresponds to P-Gibbs$_\infty$ as $\gamma =\infty$

As P-Max-Sum perfectly simulates Max-Sum, i.e., preserves the solution of its underlying non-private counterpart [18, Theorem 4.1], we now compare P-Gibbs’ quality of solution with that of Max-Sum. We begin by generating the benchmark instances.

Benchmarks. As in Sect. 6, the instances are created using pyDCOP’s generate option. We use similarly sized problems as in the experiments in Sect. 6.

• 1. Ising [23]. We generate 5 sample Ising problems with variables/agents between [10, 20) and $D_i=\{0,1\},\forall i$. The constraints are of two types: (i) binary constraints whose strength is sampled from $\mathcal {U}[\beta ,\beta ]$ where $\beta \in [1,10)$ and (ii) unary constraints whose strength is sampled from $\mathcal {U}[-\rho ,\rho ]$ where $\rho \in [0.05,0.9)$. Ising is a minimization problem.

• 2. Graph-coloring (GC). We generate 5 sample graph-coloring problems. The problems are such that the number of agents/variables lies between [50, 100] and agents’ domain size between [10, 20). Each constraint is a random integer taken from (0, 10). Graph-coloring is a minimization problem.

We omitted meeting-scheduling from this set of experiments, as Max-Sum did not perform

Results. For both Ising and GC, the solution quality remains $> 1$. That is, P-Gibbs consistently outputs better solutions than Max-Sum. The quality also improves as the privacy budget, $\epsilon$, increases from 0.046 to 9.55.

Runtime. Here, we show that while P-Max-Sum perfectly simulates Max-Sum, it does so with a significant computational overhead. More concretely, from [18, Section 6.2], we know that P-Max-Sum’s computational overhead (compared to Max-Sum), for any iteration and each node is quadratic in the domain size. E.g., from [18, Section 6.7], for random graphs, the runtime increases from 100 s for $|D|=3$, to 242 s for $|D|=5$ and 450 s for $|D|=7$.

With this, we conclude that P-Gibbs’ performance is comparable to Max-Sum (Fig. 7) and, importantly, without the significant computational overhead of P-Max-Sum.

Comparing P-Gibbs, P-RODA and P-Max-Sum. From Table 1, P-RODA [27] satisfies topology, constraint, and decision privacy. In terms of performance, P-RODA finds better quality solutions than P-Max-Sum [27]. With Fig. 7, we also see that P-Gibbs provides better quality of solution than Max-Sum (and, consequently, P-Max-Sum). Given these observations, we believe that the solutions of P-Gibbs and P-RODA may be comparable.

Furthermore, our differentially private variant is significantly less computationally expensive than P-RODA. For instance, in [27, Figure 8], we see that there is a non-linear increase in P-RODA’s runtime with an increase in the domain size. In fact, P-RODA takes (a minimum of) $\approx 200$ seconds for an iteration when the domain size is 25 [27, Figure 8]. In contrast, P-Gibbs takes $\approx$ 300 and 25 s to complete 50 iterations for randomly generated instances of graph-coloring and meeting-scheduling for the same domain size and 100 agents.

Appendix 3 Explaining P-Gibbs’privacy protection for varying $\epsilon$

To measure the proximity of the assignments between the maximum distribution values, we introduce the metric: Assignment Proximity. The critical difference between Assignment (AD) Distance and Assignment Proximity (AP) is that while AD compares the distance between the overall assignment distribution, AP compares the distance between the most probable assignment and a random assignment.

1.1 Assignment Proximity

Consider the following definition.

Definition 8

(Assignment Proximity (AP$_A$)) We define Assignment Proximity (AP$_A$) of a DCOP algorithm A as the L2-distance between the vector of the most probable assignment for each variable across l runs with the vector of the random assignment. Formally,

$$\begin{aligned} AP_A = \left\| \left( \frac{x_{f_i}:\texttt {frequent}(x_i^1,\ldots ,x_i^l)}{l}\right) _{i\in [p]}-\frac{1}{|D|}\cdot \mathbbm {1}_p\right\| _2 \end{aligned}$$

(14)

where $\mathbbm {1}_p$ is a p-dimensional unit vector, $x_i^k$ is the final assignment of variable $x_i$ in the $k\in [l]$ run, $\texttt {frequent}(\cdot )$ is a function which outputs the most frequently occurring value given an input vector and $x_{f_i}$ is the most frequent assignment of variable $x_i$.

The intuition behind introducing assignment proximity is that given AP$_{\text{ SD-Gibbs }}$ and AP$_{\text{ P-Gibbs }}$, one can compare the proximity of P-Gibbs’ assignment to a random assignment with that of SD-Gibbs. A greater value of AP$_{\text{ SD-Gibbs }}$ will imply that with SD-Gibbs, each variable is being assigned a particular domain value with high probability, in turn encoding maximum information. In contrast, a lower value of AP$_{\text{ P-Gibbs }}$ will imply that with P-Gibbs, each variable is being assigned a particular domain value with probability closer to random (i.e., 1/|D|). By comparing the assignment proximity values, we can explain the increased privacy of P-Gibbs.

1.2 Experimental Evaluation

To better visualize assignment proximity, we empirically derive the values of it for the two DCOP benchmarks, graph coloring and meeting scheduling.

Instance Setup. For graph coloring, we have 30 agents/variables with domain size 10 and each constraint between (0, 10]. For meeting scheduling, we have 30 agents/variables with domain size 20 and each constraint between $[-1000,10]{\setminus } \{0\}$.

Fig. 8

Assignment Proximity (AP) for SD-Gibbs and P-Gibbs for different $\epsilon$s, with the corresponding Solution Quality (SQ) values. The lower the AP, the higher the proximity of an algorithm’s final assignment with a perfectly random assignment. Thus, an algorithm with lower AP encodes less information regarding an agent’s utility function, preserving constraint privacy

Results. We run both the benchmark instances $l=20$ times and report the corresponding assignment proximity (AP) values in Fig. 8. From the table, observe that AP value for P-Gibbs is $\approx 50\%$ less than that for SD-Gibbs. This shows that P-Gibbs’ final assignment encodes less information compared to SD-Gibbs’, in turn, better preserving constraint privacy. Moreover, as $\epsilon$ increases, the decrease in the noise added and increase in subsampling probability results in an increase in AP values for P-Gibbs as the algorithm behaves more like SD-Gibbs.