Learning preference representations based on Choquet integrals for multicriteria decision making

Abstract

This paper concerns preference elicitation and learning of decision models in the context of multicriteria decision making. We propose an approach to learn a representation of preferences by a non-additive multiattribute utility function, namely a Choquet or bi-Choquet integral. This preference model is parameterized by one-dimensional utility functions measuring the attractiveness of consequences w.r.t. various point of views and one or two set functions (capacities) used to weight the coalitions and control the intensity of interactions among criteria, on the positive and possibly the negative sides of the utility scale. Our aim is to show how we can successively learn marginal utilities from properly chosen preference examples and then learn where the interactions matter in the overall model. We first present a preference elicitation method to learn spline representations of marginal utilities on every component of the model. Then we propose a sparse learning approach based on adaptive $L_1$-regularization for determining a compact Möbius representation fitted to the observed preferences. We present numerical tests to compare different regularization methods. We also show the advantages of our approach compared to basic methods that do not seek sparsity or that force sparsity a priori by requiring k-additivity.

Appendices

Appendix A

In this appendix we explain how equations on utilities (5)–(6) and (7)–(10) are derived from preference and indifference statements. The explanation is directly given for the bi-CIU model but can be adapted to the particular case of the CIU model by taking $w=v$.

1.1 A.1   Utility elicitation with solvability

1.1.1 A.1.1   Utility elicitation below the neutral level with solvability assumption

Proposition 1

Let $x_i,h_i \in X_i$ such that $x_i \precsim _i \textbf{0}_i$ and $h_i \precsim _i \textbf{0}_i$ and $r_j$, $R_j \in X_j$ such that $\textbf{0}_j \precsim _j r_j \prec _j R_j$. Suppose that $(x_i, r_j, \textbf{0}_{-ij}) \sim (y_i, R_j, \textbf{0}_{-ij})$ and $(h_i, r_j, \textbf{0}_{-ij}) \sim (z_i, R_j, \textbf{0}_{-ij})$. Assuming $(\mathbf{-1}_i, \textbf{0}_{-i}) \prec \textbf{0}$, we have:

$$\begin{aligned} u_i(y_i) - u_i(x_i) = u_i(z_i) - u_i(y_i) \end{aligned}$$

Proof

From $(x_i, r_j, \textbf{0}_{-ij}) \!\!\sim \!\! (y_i, R_j, \textbf{0}_{-ij})$ we have: $f^u_{v,w}(x_i, r_j, \textbf{0}_{-ij}) \!\!=\!\! f^u_{v,w}(y_i, R_j, \textbf{0}_{-ij})$. Also, since $r_j \prec _j R_j$ we have $y_i \prec _i x_i$. Moreover, since $x_i \precsim _i \textbf{0}_i$ and $\textbf{0}_j \precsim _j r_j \prec _j R_j$, we have $u_i(y_i)< u_i(x_i) \le 0 \le u_j(r_j) < u_j(R_j)$ and therefore $f^u_{v,w}(x_i, r_j, \textbf{0}_{-ij}) = u_j(r_j)v(\{j\}) + u_i(x_i) (1 - w(N\setminus \{i\}))$.

Similarly, we have: $f^u_{v,w}(y_i, R_j, \textbf{0}_{-ij}) = u_j(R_j)v(\{j\}) + u_i(y_i) (1 - w(N\setminus \{i\}))$. Hence we have: $(u_i(x_i) - u_i(y_i)) (1 - w(N\setminus \{i\})) = (u_j(R_j)-u_j(r_j))v(\{j\})$.

Moreover, using the second indifference $(h_i, r_j, \textbf{0}_{-ij}) \sim (z_i, R_j, \textbf{0}_{-ij})$, we obtain $(u_i(h_i) - u_i(z_i)) (1 - w(N\setminus \{i\})) = (u_j(R_j)-u_j(r_j))v(\{j\})$. Then $(u_i(x_i) - u_i(y_i)) (1 - w(N\setminus \{i\})) = (u_i(h_i) - u_i(z_i)) (1 - w(N\setminus \{i\}))$. Assuming $(\mathbf{-1}_i, \textbf{0}_{-i}) \prec \textbf{0}$, i.e., $w(N\setminus \{i\})< 1$ we obtain:

$$\begin{aligned} u_i(x_i) - u_i(y_i) = u_i(h_i) - u_i(z_i) \end{aligned}$$

$\square$

1.1.2 A.1.2   Utility elicitation above the neutral level with solvability assumption

Proposition 2

Let $x_i,h_i \in X_i$ such that $x_i \succsim \textbf{0}_i$ and $h_i \succsim \textbf{0}_i$ and $r_j,R_j \in X_j$ and $x_i \in X_i$ such that $r_j \prec _j R_j \precsim _j\textbf{0}_{j}$. Suppose that $(x_i, R_j, \textbf{0}_{-ij}) \sim (y_i, r_j, \textbf{0}_{-ij})$ and $(h_i, R_j, \textbf{0}_{-ij}) \sim (z_i, r_j, \textbf{0}_{-ij})$. Assuming $(\textbf{1}_i,\textbf{0}_{-i}) \succ \textbf{0}$, we have:

$$\begin{aligned} u_i(x_i) - u_i(y_i) = u_i(h_i) - u_i(z_i) \end{aligned}$$

Proof

From $(x_i, R_j, \textbf{0}_{-ij}) \sim (y_i, r_j, \textbf{0}_{-ij})$, we have: $f^u_{v,w}(x_i, R_j, \textbf{0}_{-ij}) = f^u_{v,w}(y_i, r_j, \textbf{0}_{-ij})$. Since $r_j \prec _j R_j$ we have $x_i \prec _i y_i$. Moreover, since $x_i \succsim _i \textbf{0}_i$ and $r_j \prec _j R_j \precsim _j\textbf{0}_{j}$, we have $u_j(r_j)< u_j(R_j) \le 0 \le u_i(x_i) < u_i(y_i)$ and therefore $f^u_{v,w}(x_i, R_j, \textbf{0}_{-ij}) = u_i(x_i)v(\{i\}) + u_j(R_j) (1 - w(N\setminus \{j\}))$.

Similarly $f^u_{v,w}(y_i, r_j, \textbf{0}_{-ij}) = u_i(y_i)v(\{i\}) + u_j(r_j) (1 - w(N\setminus \{j\})).$ Hence we have: $(u_i(y_i) - u_i(x_i)) v(\{i\}) = (u_j(R_j)-u_j(r_j))(1 -w(N\setminus \{j\}))$.

Moreover, using the second indifference $(h_i, R_j, \textbf{0}_{-ij}) \sim (z_i, r_j, \textbf{0}_{-ij})$, we obtain $(u_i(z_i) - u_i(h_i)) v(\{i\}) = (u_j(R_j)-u_j(r_j))(1 -w(N\setminus \{j\}))$. Then $(u_i(y_i) - u_i(x_i)) v(\{i\}) = (u_i(z_i) - u_i(h_i)) v(\{i\})$. Assuming $(\textbf{1}_i,\textbf{0}_{-i}) \succ \textbf{0}$, i.e., $v(\{i\})>0$ we obtain:

$$\begin{aligned} u_i(y_i) - u_i(x_i) = u_i(z_i) - u_i(h_i) \end{aligned}$$

$\square$

1.2 A.2   Utility elicitation without solvability

1.2.1 A.2.1   Utility elicitation below the neutral level without solvability assumption

Proposition 3

Assume that the elements of $X_i$ are denoted $x_{i,k}$ and indexed according to their relative values: $x_{i,k} \precsim _i x_{i,k+1}$, for any k. Let $x_i,h_i \in X_i$ such that $x_i \precsim _i \textbf{0}_i$ and $h_i \precsim _i \textbf{0}_i$ and $r_j$, $R_j \in X_j$ such that $\textbf{0}_j \precsim _j r_j \prec _j R_j$. Let k be the lower integer such that $(x_i, r_j, \textbf{0}_{-ij}) \precsim (x_{i,k+1}, R_j, \textbf{0}_{-ij})$ and $k'$ the highest integer such that $(h_i, r_j, \textbf{0}_{-ij}) \succsim (x_{i,k'}, R_j, \textbf{0}_{-ij})$. Assuming $(\mathbf{-1}_i, \textbf{0}_{-i}) \prec \textbf{0}$, we have:

$$\begin{aligned} u_i(h_i) - u_i(z_i^-)\ge & {} u_i(x_i) - u_i(y_i^+) \\ u_i(h_i) - u_i(z_i^+)< & {} u_i(x_i) - u_i(y_i^-) \end{aligned}$$

where $y_i^+ = x_{i,k+1}$, $y_i^- = x_{i,k}$, $z_i^- = x_{i,k'}$ and $z_i^+ = x_{i,k'+1}$.

Proof

By construction $y_i^-$ necessarily verifies the following strict preference: $(x_i, r_j, \textbf{0}_{-ij}) \succ (y_{i}^-, R_j, \textbf{0}_{-ij})$. Hence, with $(x_i, r_j, \textbf{0}_{-ij}) \precsim (y_i^{+}, R_j, \textbf{0}_{-ij})$, we obtain the following inequations: $(u_i(x_i) - u_i(y_i^+)) (1 - w(N\setminus \{i\})) \le (u_i(R_j)-u_i(r_j))v(\{j\})$ and $(u_i(x_i) - u_i(y_{i}^-)) (1 - w(N\setminus \{i\})) > (u_i(R_j)-u_i(r_j))v(\{j\})$.

Similarly, $z_i^+$ verify $(h_i, r_j, \textbf{0}_{-ij}) \prec (z_i^+, R_j, \textbf{0}_{-ij})$. Hence, with $(h_i, r_j, \textbf{0}_{-ij}) \succsim (z_i^-, R_j, \textbf{0}_{-ij})$ we obtain the following inequations: $(u_i(h_i) - u_i(z_i^-) (1 - w(N\setminus \{i\})) \ge (u_i(R_j)-u_i(r_j))v(\{j\})$ and $(u_i(h_i) - u_i(z_i^+)) (1 - w(N\setminus \{i\})) < (u_i(R_j)-u_i(r_j))v(\{j\})$. Hence we have $(u_i(x_i) - u_i(y_i^+))(1 - w(N\setminus \{i\})) \le (u_i(R_j)-u_i(r_j))v(\{j\}) \le (u_i(h_i) - u_i(z_i^-)) (1 - w(N\setminus \{j\}))$.

Moreover, $(u_i(h_i) - u_i(z_i^+) (1 - w(N\setminus \{i\}))<(u_i(R)-u_i(r))v(\{j\}) < (u_i(x_i) - u_i(y_i^-) (1 - w(N\setminus \{i\}))$. Assuming $(\mathbf{-1}_i, \textbf{0}_{-i}) \prec \textbf{0}$, i.e., $w(N\setminus \{i\})< 1$, we obtain:

$$\begin{aligned} u_i(h_i) - u_i(z_i^-)\ge & {} u_i(x_i) - u_i(y_i^+) \\ u_i(h_i) - u_i(z_i^+)< & {} u_i(x_i) - u_i(y_i^-) \end{aligned}$$

$\square$

1.2.2 A.2.2   Utility elicitation above the neutral level without solvability assumption

Proposition 4

Assume that the elements of $X_i$ are denoted $x_{i,k}$ and indexed according to their relative values: $x_{i,k} \precsim _i x_{i,k+1}$, for any k. Let $x_i,h_i \in X_i$ such that $x_i \succsim _i \textbf{0}_i$ and $h_i \succsim _i \textbf{0}_i$ and $r_j$, $R_j \in X_j$ such that $r_j \prec _j R_j \precsim _j\textbf{0}_{j}$. Let k be the higher integer such that $(x_i, R_j, \textbf{0}_{-ij}) \succsim (x_{i,k}, r_j, \textbf{0}_{-ij})$ and $k'$ the lower integer such that $(h_i, R_j, \textbf{0}_{-ij}) \precsim (x_{i,k+1}, r_j, \textbf{0}_{-ij})$. Assuming $(\textbf{1}_i,\textbf{0}_{-i}) \succ \textbf{0}$, we have:

$$\begin{aligned} u_i(h_i) - u_i(z_i^+)\le & {} u_i(x_i)- u_i(y_i^-) \\ u_i(h_i) - u_i(z_i^-)> & {} u_i(x_i) -u_i(y_i^+) \end{aligned}$$

where $y_i^+ = x_{i,k+1}$, $y_i^- = x_{i,k}$, $z_i^- = x_{i,k'}$ and $z_i^+ = x_{i,k'+1}$.

Proof

By construction $y^{+}_i$ necessarily verifies the following strict preference: $(x_i, R_j, \textbf{0}_{-ij}) \prec (y^{+}_i, r_j, \textbf{0}_{-ij})$. Hence, with $(x_i, R_j, \textbf{0}_{-ij}) \succsim (y_i^-, r_j, \textbf{0}_{-ij})$, we obtain the following inequations: $(u_i(x_i) - u_i(y_i^-) )v(\{i\}) \ge (u_i(r_j)-u_i(R_j)) (1 - w(N\setminus \{j\}))$ and $(u_i(x_i) - u_i(y_i^+) )v(\{i\}) < (u_i(r_j)-u_i(R_j)) (1 - w(N\setminus \{j\}))$.

Similarly $z_i^-$ verify $(h_i, R_j, \textbf{0}_{-ij}) \succ (z^{-}_i, r_j, \textbf{0}_{-ij})$. Hence, with $(h_i, R_j, \textbf{0}_{-ij}) \precsim (z_{i}^+, r_j, \textbf{0}_{-ij})$, we obtain the following inequations: $(u_i(h_i) - u_i(z_i^+) )v(\{i\}) \le (u_i(r_j)-u_i(R_j)) (1 - w(N\setminus \{j\}))$ and $(u_i(h_i) - u_i(z_i^-) )v(\{i\}) > (u_i(r_j)-u_i(R_j)) (1 - w(N\setminus \{j\}))$. Hence we have $(u_i(h_i) - u_i(z_i^+) )v(\{i\}) \le (u_i(R_j)-u_i(r_j)) (1 - w(N\setminus \{j\})) \le ( u_i(x_i)- u_i(y_i^-) )v(\{i\})$. Moreover, $(u_i(x_i) -u_i(y_i^+) )v(\{i\})< (u_i(R_j)-u_i(r_j)) (1 - w(N\setminus \{j\})) < (u_i(h_{i}) - u_i(z_i^-) )v(\{i\})$. Assuming $(\textbf{1}_i,\textbf{0}_{-i}) \succ \textbf{0}$, i.e., $v(\{i\})>0$, we obtain:

$$\begin{aligned} u_i(h_i) - u_i(z_i^+)\le & {} u_i(x_i)- u_i(y_i^-) \\ u_i(h_i) - u_i(z_i^-)> & {} u_i(x_i) -u_i(y_i^+) \end{aligned}$$

$\square$

Appendix B

In the following, we use the convention that for any subset $B \subseteq N$, $\int _{\mathcal {I}_B} f(z_1, \ldots ,z_n)dz_B$ denotes the multiple integral of the function f w.r.t. the arguments $z_i,i \in B$ on the hypercube $\mathcal {I}_B = [0,1]^{|B |}$.

Lemma 5

Let $B \subseteq N \setminus \emptyset$ and $k \in \mathbb {N}^*$, then the following equality holds:

$$\begin{aligned} \int _{\mathcal {I}_B} \min _{i \in B }\{z_i\}^kdz_B = \frac{k!}{ \prod _{i=1}^k( |B |+i)} \end{aligned}$$

Proof

Consider $|B |$ random variables $(Z_i)_{i\in B}$ independent and identically distributed according a uniform distribution within [0, 1]. It can be easily shown that the random variable $Y = \min _{i\in B}\{Z_i\}$ admits the following density function:

$$f_Y(y) = {\left\{ \begin{array}{ll} |B |(1-y)^{|B |-1} \text { if } y \in [0,1]\\ 0 \text { else. } \end{array}\right. }$$

Then we obtain:

$$\begin{aligned} \mathbb {E}[ Y^k] =\int _0^1y^k|B |(1-y)^{|B |-1}dy&= k \int _0^1 y^{k-1}(1-y)^{|B |}dy\\&= k(k-1) \int _0^1 y^{k-2}\frac{(1-y)^{|B |+1}}{|B |+1}dy \\ {}&=\ldots \\ {}&=k! \int _0^1y^{k-k}\frac{(1-y)^{|B |+k-1}}{\prod _{i=1}^{k-1}( |B |+i)}dy \\ {}&=\frac{k!}{\prod _{i=1}^{k-1}( |B |+i)}\frac{1}{(|B |+ k)} = \frac{k!}{\prod _{i=1}^{k}( |B |+i)} \end{aligned}$$

Finally, we conclude:

$$ \int _{\mathcal {I}_B} \min _{i \in B }\{z_i\}^kdz_B = \mathbb {E}[ \min _{i \in B}\{Z_i \}^k] =\mathbb {E}[ Y^k] = \frac{k!}{\prod _{i=1}^{k}( |B |+i)}$$

$\square$

Lemma 6

Let $n \le 3$ and $B_1,B_2 \subseteq N$ such that $B_1 \cap B_2 = \emptyset$ and $B_2 \ne \emptyset$. For any vector $(z_{j})_{j\in B_2}$ taking values in [0, 1], the following equality holds:

$$\begin{aligned} \int _{\mathcal {I}_{B_1}} \min _{i \in B_2 \cup B_1}\{z_i\} dz_{B_1} = \wedge _{B_2}- \frac{|B_1 |\wedge _{B_2}^2}{2} + \frac{|B_1 |(|B_1 |-1)^+\wedge _{B_2}^3}{6} \end{aligned}$$

with $\wedge _{B_2} = \min _{i \in B_2 }\{z_i\}$ and $x^+ = \max \{0,x\}$.

Proof

Firstly, for any $A\subseteq N\setminus \emptyset$, any vector $(z_i)_{i \in A}$ valued in [0, 1] and any $k \in \mathbb {N}$, we have:

$$\begin{aligned} \int _0^1 \min \{x, \wedge _A\}^kdx&= \int _0^{\wedge _A} \min \{x, \wedge _A\}^kdx+ \int _{\wedge _A}^ 1 \min \{x, \wedge _A\}^kdx \nonumber \\ {}&=\int _0^{\wedge _A} x^k dx+ \int _{\wedge _A}^ 1 \wedge _A^k = \frac{\wedge _A^{k+1}}{k+1} +(1-\wedge _A)\wedge _A^k \nonumber \\ {}&= \wedge _A^k -\frac{k}{k+1}\wedge _A^{k+1} \end{aligned}$$

(1)

where $\wedge _A =\min _{i \in A}\{z_i\}$.

Remark that for $B_1 = \emptyset$, the left-hand term boils down to $\wedge _{B_2}$ which equals the right-hand term for $|B_1 |= 0$. Suppose now that $B_1 \ne \emptyset$. Since $n \le 3$, $B_2 \ne \emptyset$ and $B_1 \cap B_2 = \emptyset$, $B_1$ is necessarily a singleton or a pair, then we have: $|B_1 |\in \{1, 2\}$.

Then let $(\pi _1, \ldots , \pi _{|B_1 |})$ be any ordering of the elements of $B_1$. Using (B1) with $A = (B_1 \cup B_2)\setminus \{ \pi _1\}$, $x=z_{\pi _1}$ and $k=1$, we have:

$$\begin{aligned}&\int _{\mathcal {I}_{B_1}} \min _{i \in B_2 \cup B_1}\{z_i\}dz_{B_1} = \int _{\mathcal {I}_{ B_1\setminus \{\pi _1\}}} \Big ( \int _0^1 \min _{i \in B_2 \cup B_1}\{z_i\} dz_{\pi _1}\Big )dz_{ B_1\setminus \{\pi _1\}}\\&= \int _{\mathcal {I}_{ B_1\setminus \{\pi _1\}}} \Big (\wedge _{(B_1 \cup B_2 )\setminus \{ \pi _1\}} -\frac{\wedge _{(B_1 \cup B_2 )\setminus \{ \pi _1\}}^2}{2}\Big )dz_{ B_1\setminus \{\pi _1\}} \end{aligned}$$

Then if $|B_1 |= 1$, we have $B_1 \setminus \{ \pi _1 \} = \emptyset$ and therefore we obtain:

$$\begin{aligned} \int _{\mathcal {I}_{B_1}} \min _{i \in B_2 \cup B_1}\{z_i\} dz_{B_1}&= \wedge _{(B_1 \cup B_2)\setminus \{ \pi _1\}} -\frac{\wedge _{(B_1 \cup B_2) \setminus \{ \pi _1\}}^2}{2} \\ {}&=\wedge _{B_2}- \frac{|B_1 |\wedge _{B_2}^2}{2} + \frac{|B_1 |(|B_1 |-1)^+\wedge _{B_2}^3}{6} \end{aligned}$$

Finally, if $|B_1 |= 2$, we have $B_1 \setminus \{ \pi _1 \} = \{ \pi _2 \}$ and using (B1) for $A = (B_1 \cup B_2) \setminus \{ \pi _1,\pi _2\} = B_2$, $x=z_{\pi _2}$ and $k\in \{1,2\}$, we obtain:

$$\begin{aligned} \int _{\mathcal {I}_{B_1}} \min _{i \in B_2 \cup B_1}\{z_i\} dz_{B_1}&= \int _0^1 \Big (\wedge _{(B_1 \cup B_2) \setminus \{ \pi _1\}} -\frac{\wedge _{(B_1 \cup B_2) \setminus \{ \pi _1\}}^2}{2}\Big )dz_{\pi _2} \\ {}&= \int _0^1 \Big (\min _{i \in B_2 \cup \{ \pi _2\}}\{z_i\} -\frac{\min _{i \in B_2 \cup \{ \pi _2\}}\{z_i\}^2}{2}\Big )dz_{\pi _2} \\ {}&= \wedge _{B_2}- \frac{|B_1 |\wedge _{B_2}^2}{2} + \frac{|B_1 |(|B_1 |-1)^+\wedge _{B_2}^3}{6} \end{aligned}$$

$\square$

Proposition 7

Let $n \le 3$. Assume that the features $(\phi _j)_{j=1}^{2^n}$ are defined such that $\phi _j = \min _{ i \in \rho ^{-1}(j)}\{Z_i\}$ where $(Z_i)_{i \in N}$ are i.i.d. random variables such that $Z_i \sim \mathcal {U}([0,1])$ for any $i \in N$. Then, for any pair of criteria coalition $S_1,S_2 \subseteq N \setminus \emptyset$ of cardinals $|S_1|= s_1$,$|S_2|= s_2$ and $|S_1 \cap S_2|= s_{12} \ne 0$, the covariance between $\phi _{\rho (S_1)}$ and $\phi _{\rho (S_2)}$ reads as follows:

$$\begin{aligned} Cov(\phi _{\rho (S_1)},\phi _{\rho (S_2)}) =\sum _{k=1}^3g_k(s_{12})\gamma _k(s_1,s_2,s_{12})-\frac{1}{(s_1 + 1)(s_2 + 1)} \end{aligned}$$

with $g_k(s_{12}) = \frac{k!}{\prod _{i=1}^k(s_{12}+i)}$, $\gamma _1= 1$, $\gamma _2(s_1,s_2,s_{12}) =-\frac{1}{2}((s_1-s_{12})^++(s_2-s_{12})^+)$ and $\gamma _3(s_1,s_2,s_{12}) = \frac{1}{4}((s_1-s_{12})^+(s_2-s_{12})^+ ) +\frac{1}{6}((s_1-s_{12})^+(s_1-s_{12}-1)^++(s_2-s_{12})^+(s_2-s_{12}-1)^+)$ where $x^+ = \max \{x, 0\}$.

Proof

Let $S_1, S_2 \subseteq N \setminus \emptyset$ such that $S_1 \cap S_2 \ne 0$. Firstly, recall that:

$$ Cov(\phi _{\rho (S_1)},\phi _{\rho (S_2)}) = \mathbb {E}[\phi _{\rho (S_1)} \phi _{\rho (S_2)} ] - \mathbb {E}[\phi _{\rho (S_1)}]\mathbb {E}[\phi _{\rho (S_2)}]$$

Also, since the variables $(Z_i)_{i \in N}$ are independent and identically distributed according a uniform distribution within [0, 1], for any $S \subseteq N$, we have:

$$\begin{aligned} \mathbb {E}[\phi _{\rho (S)}]= \int _{\mathcal {I}_S} \min _{i \in S}\{z_i\} dz_S \end{aligned}$$

Then using Lemma 5 for $k=1$ and $B =S$, we obtain:

$$\begin{aligned} \mathbb {E}[\phi _{\rho (S)}]= \frac{1}{|S |+ 1} \end{aligned}$$

(2)

Moreover, since $S_1 \cap S_2 \ne \emptyset$, we have:

$$\begin{aligned}&\mathbb {E}[\phi _{\rho (S_1)} \phi _{\rho (S_2)}] = \mathbb {E}[\min _{ i \in S_1}\{Z_i\}\min _{ i \in S_2}\{Z_i\}]\\&= \int _{\mathcal {I}_{S_1 \cup S_2}} \min _{i \in S_1}\{z_i\} \min _{i \in S_2}\{z_i\} dz_{ S_1 \cup S_2} \\&= \int _{\mathcal {I}_{S_2}} \min _{i \in S_2}\{z_i\} \Big ( \int _{\mathcal {I}_{S_1 \setminus S_1 \cap S_2}}\min _{i \in S_1}\{z_i\} dz_{ S_1 \setminus (S_1 \cap S_2) }\Big ) dz_{S_2} \end{aligned}$$

Then, using Lemma 6 sequentially for $B_1 = S_1 \setminus (S_1 \cap S_2)$, $B_2 = S_1 \cap S_2$ and $B_1 = S_2\setminus (S_1 \cap S_2)$, $B_2 = S_1 \cap S_2$, we obtain:

$$\begin{aligned}&\mathbb {E}[\phi _{\rho (S_1)} \phi _{\rho (S_2)}] = \int _{\mathcal {I}_{S_1 \cap S_2}}\Big ( \int _{\mathcal {I}_{S_2 \setminus (S_1 \cap S_2)}}\min _{i \in S_2}\{z_i\} \Big (\wedge _{S_1 \cap S_2}-\frac{(s_1-s_{12})^+\wedge _{S_1 \cap S_2}^2}{2}\\ {}&+\frac{(s_1-s_{12})^+(s_1-s_{12}-1)^+\wedge _{S_1 \cap S_2}^3}{6} \Big )dz_{ S_2 \setminus (S_1 \cap S_2)} \Big ) dz_{S_1 \cap S_2} \\ {}&= \int _{\mathcal {I}_{S_1 \cap S_2}} \Big (\wedge _{S_1 \cap S_2}-\frac{(s_1-s_{12})^+\wedge _{S_1 \cap S_2}^2}{2}+\frac{(s_1-s_{12})^+(s_1-s_{12}-1)^+\wedge _{S_1 \cap S_2}^3}{6} \Big )\\&\Big (\wedge _{S_1 \cap S_2}-\frac{(s_2-s_{12})^+\wedge _{S_1 \cap S_2}^2}{2}+\frac{(s_2-s_{12})^+(s_2-s_{12}-1)^+\wedge _{S_1 \cap S_2}^3}{6} \Big ) dz_{S_1 \cap S_2} \\ \end{aligned}$$

where $\wedge _{S_1 \cap S_2} = \min _{i \in S_1 \cap S_2}\{z_i\}$ for any vector $(z_i)_{i \in S_1 \cap S_2}$ valued in [0, 1]. This expression can be simplified remarking that since $n \le 3$ and $S_1 \cap S_2 \ne \emptyset$, we have that the cross products $(s_2-s_{12})^+(s_2-s_{12}-1)^+(s_1-s_{12})^+(s_1-s_{12}-1)^+$, $(s_2-s_{12})^+(s_2-s_{12}-1)^+(s_1-s_{12})^+$ and $(s_1-s_{12})^+(s_1-s_{12}-1)^+(s_2-s_{12})^+$ necessarily equal zero. Finally, using Lemma 5 for $B = S_1 \cap S_2$ and $k \in \{2,3,4\}$, we obtain that:

$$\begin{aligned}&\mathbb {E}[\phi _{\rho (S_1)} \phi _{\rho (S_2)}] = g_2(s_{12})-g_3(s_{12})\frac{1}{2}((s_1-s_{12})^++(s_2-s_{12})^+) \nonumber \\ {}&+g_4(s_{12})(\frac{1}{4}((s_1-s_{12})^+(s_2-s_{12})^+ ) \nonumber \\ {}&+\frac{1}{6}((s_1-s_{12})^+(s_1-s_{12}-1)^++(s_2-s_{12})^+(s_2-s_{12}-1)^+)) \end{aligned}$$

(3)

with $g_k(s_{12}) = \frac{k!}{\prod _{i=1}^k(s_{12}+i)}$. We obtain the final result by combining (B3) and (B2). $\square$

Proposition 8

Suppose that $n=3$ and that the underlying model $\beta ^*$ is such that $A_1= \{\{1\},\{ 2 \}, \{3\}, \{ 1,2,3 \}\}$, $A_2 = \{\{ 1,2\},\{ 1,3\}, \{ 2,3\}\}$ and ${{\,\textrm{sign}\,}}(\beta _{A_1}^*) = { \textbf{1}}$. Assume also that the features $(\phi _j)_{j=1}^{2^n}$ are defined such that $\phi _j = \min _{ i \in \rho ^{-1}(j)}\{Z_i\}$ where $(Z_i)_{i \in N}$ are i.i.d. random variables such that $Z_i \sim \mathcal {U}([0,1])$ for any $i \in N$. Then, Condition (19) boils down to:

$$\begin{aligned} |2V_{1,2}^1(V_{3,3}^3-V_{1,3}^1)+V_{2,3}^2(V_{1,1}^1-3V_{1,3}^1) |< |V_{3,3}^3V_{1,1}^1-3(V_{1,3}^1)^2|\end{aligned}$$

where $V_{j,k}^{l} = Cov(\phi _{\rho (S_1)},\phi _{\rho (S_2)})= C_{\rho (S_1),\rho (S_2)}$ for any pair of criteria coalition $S_1,S_2 \subseteq N$ such that $j = |S_1 |$, $k = |S_2 |$ and $l = |S_1 \cap S_2 |$.

Proof

Since the random variables $(Z_i)_{i \in N}$ are supposed independent, for any pair of criteria coalition $S_1, S_2 \subseteq N$ such that $S_1 \cap S_2 = \emptyset$, we have: $Cov(\phi _{\rho (S_1)},\phi _{\rho (S_2)})= 0$. Also, indexing the columns and rows of $C_{11}$ and $C_{21}$ in the lexicographical order, we have:

$$C_{11} = \begin{pmatrix} V_{1,1}^1&{}0&{}0&{}V_{1,3}^1 \\ 0&{}V_{1,1}^1&{}0&{}V_{1,3}^1\\ 0&{}0&{}V_{1,1}^1&{}V_{1,3}^1 \\ V_{1,3}^1&{}V_{1,3}^1&{}V_{1,3}^1&{}V_{3,3}^3 \end{pmatrix} \text { and } C_{21} = \begin{pmatrix} V_{1,2}^1 &{} V_{1,2}^1 &{} 0 &{} V_{2,3}^2 \\ V_{1,2}^1&{} 0 &{} V_{1,2}^1&{}V_{2,3}^2\\ 0&{}V_{1,2}^1&{} V_{1,2}^1 &{}V_{2,3}^2 \\ \end{pmatrix} $$

Then $C_{11}$ can be rewritten as a block-matrix as follows:

$$ C_{11} = \begin{pmatrix} M_{1}&{}M_2^T \\ M_2&{}M_3 \\ \end{pmatrix}\text { with } M_1 = \begin{pmatrix} V_{1,1}^1&{}0&{}0 \\ 0&{}V_{1,1}^1&{}0\\ 0&{}0&{}V_{1,1}^1 \\ \end{pmatrix}, M_2 = \begin{pmatrix} V_{1,3}^1&V_{1,3}^1&V_{1,3}^1 \end{pmatrix}, M_3 =\begin{pmatrix} V_{3,3}^3 \end{pmatrix} $$

Since $C_{11}$ is the covariance matrix of the random variables $(\phi _j)_{j \in A_1}$ it is a positive semi-definite matrix. Also, remarking that $M_1$ is inversible and using Proposition 5, one can compute the Schur complement $S = M_3 - M_2^TM_1^{-1}M_2 = \frac{V_{3,3}^3V_{1,1}^1-3V_{1,3}^1}{V_{1,1}^1} \ne 0$ and obtain that $C_{11}$ is a positive definite matrix the inverse of which reads as follows:

$$\begin{aligned} C_{11}^{-1}&= \begin{pmatrix} M_1^{-1}+M_1^{-1}M_2^TS^{-1}M_2M_1^{-1}&{} -M_1^{-1}M_2^TS^{-1} \\ -S^{-1}M_2M_1^{-1}&{}S^{-1} \end{pmatrix}\\&= \frac{1}{SV_{1,1}^1} \begin{pmatrix} S + \frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}\frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}\frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}-V_{1,3}^1 \\ \frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}S + \frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}\frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}-V_{1,3}^1\\ \frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}\frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}S + \frac{(V_{1,3}^1)^2}{V_{1,1}^1}&{}-V_{1,3}^1\\ -V_{1,3}^1 &{} - V_{1,3}^1 &{} -V_{1,3}^1 &{} V_{1,1}^1 \end{pmatrix} \end{aligned}$$

Then we finally obtain:

$$C_{21}C_{11}^{-1}{{\,\textrm{sign}\,}}(\beta ^*_{A_1}) = \frac{2V_{1,2}^1(V_{3,3}^3-V_{1,3}^1)+V_{2,3}^2(V_{1,1}^1-3V_{1,3}^1)}{V_{3,3}^3V_{1,1}^1-3(V_{1,3}^1)^2} \begin{pmatrix} 1 \\ 1 \\ 1\\ \end{pmatrix}$$

Therefore Condition (19) is satisfied if and only if $|2V_{1,2}^1(V_{3,3}^3-V_{1,3}^1)+V_{2,3}^2(V_{1,1}^1-3V_{1,3}^1) |< |V_{3,3}^3V_{1,1}^1-3(V_{1,3}^1)^2|$. $\square$