Randomized Lagrangian stochastic approximation for large-scale constrained stochastic Nash games

Abstract

In this paper, we consider stochastic monotone Nash games where each player’s strategy set is characterized by possibly a large number of explicit convex constraint inequalities. Notably, the functional constraints of each player may depend on the strategies of other players, allowing for capturing a subclass of generalized Nash equilibrium problems (GNEP). While there is limited work that provide guarantees for this class of stochastic GNEPs, even when the functional constraints of the players are independent of each other, the majority of the existing methods rely on employing projected stochastic approximation (SA) methods. However, the projected SA methods perform poorly when the constraint set is afflicted by the presence of a large number of possibly nonlinear functional inequalities. Motivated by the absence of performance guarantees for computing the Nash equilibrium in constrained stochastic monotone Nash games, we develop a single timescale randomized Lagrangian multiplier stochastic approximation method where in the primal space, we employ an SA scheme, and in the dual space, we employ a randomized block-coordinate scheme where only a randomly selected Lagrangian multiplier is updated. We show that our method achieves a convergence rate of $\mathcal {O}\left( \frac{\log (k)}{\sqrt{k}}\right)$ for suitably defined suboptimality and infeasibility metrics in a mean sense.

Appendix

1.1 Proof of Lemma 1

Proof

Invoking Proposition 1 and taking into account that $\mathcal {N}_X(x^*) = \partial \mathcal {I}_X(x^*)$, we have that $x^* \in X$ solves the following augmented variational inequality problem $\text{ VI }\left( X,F+J^{-1}\nabla f^T\lambda ^*\right) ,$ that is parameterized by J and $\lambda ^*$. This implies that

$$\begin{aligned} \left( F(x^*) +J^{-1}\nabla f(x^*)^T\lambda ^*\right) ^T(x-x^*) \ge 0, \quad \hbox {for all}\quad x \in X. \end{aligned}$$

(18)

From the convexity of function $f_j$ for all $j \in [J]$ and that $\lambda _j \ge 0$, we have

$$\begin{aligned} {\lambda _j^*\left( f_j(x)-f_j(x^*)\right) }\ge \lambda _j^*\nabla f_j(x^*)^T(x-x^*). \end{aligned}$$

Summing the preceding relation over $j \in [J]$ and recalling the definition of the mapping f(x), we obtain

$$\begin{aligned} \left( f(x) -f(x^*)\right) ^T\lambda ^* \ge \left( \nabla f(x^*)^T\lambda ^*\right) ^T(x-x^*). \end{aligned}$$

Invoking Proposition 1(ii) we obtain $f(x)^T\lambda ^* \ge \left( \nabla f(x^*)^T\lambda ^*\right) ^T(x-x^*)$. From the preceding relation and (18) we obtain $F(x^*) ^T(x-x^*) +J^{-1}f(x)^T\lambda ^* \ge 0$ for all $x \in X$. $\square$

1.2 Proof of Lemma 2

Proof

(i) Note that $x^*$ is a feasible point to problem (cSVI) with respect to the set $\mathcal {X}$, i.e., $x^* \in \mathcal {X}$. Also, note that ${{\hat{\lambda }} \ge 0}$. From the definition of $\Phi _\rho$, we have that $\Phi _\rho (x^*,{\hat{\lambda }})\le 0$. Let $x:=x^*$ in (4). Then we have

$$\begin{aligned} {F(x^*) ^T}({\hat{x}}-x^*) +J^{-1}f({\hat{x}})^T\lambda \le {C(x^*,\lambda )}. \end{aligned}$$

(19)

Also, from Lemma 1 and that ${\hat{x}} \in X$ we have

$$\begin{aligned} {0 \le F(x^*) ^T(\hat{x}-x^*) +J^{-1}f(\hat{x})^T\lambda ^*.} \end{aligned}$$

The preceding relation and that $\lambda ^* \ge 0$ imply that

$$\begin{aligned} 0 \le {F(x^*) ^T}({\hat{x}}-x^*) +J^{-1}[f({\hat{x}})]_+^T\lambda ^*. \end{aligned}$$

Summing the preceding relation and (19) and rearranging the terms, we obtain

$$\begin{aligned} J^{-1}f({\hat{x}})^T\lambda -J^{-1}[f({\hat{x}})]_+^T\lambda ^* \le {C(x^*,\lambda )}. \end{aligned}$$

(20)

Let us choose $\lambda _j:=1+\lambda ^*_j$ if $f_j({\hat{x}})>0$, and $\lambda _j:=0$ otherwise for all $j \in [J]$. Then, we obtain the desired relation in (i).

(ii) Let $\lambda =0$ in (4) and note that $\Phi _\rho (x,{\hat{\lambda }})\le 0$ for all $x\in \mathcal X$. We have $F(x)^T({\hat{x}}-x)\le C(x,0)$ for all $x\in \mathcal X$. Taking supremum from the both sides, we obtain desired results in (ii). $\square$

1.3 Proof of Lemma 3

Proof

The relations in part (i) hold as a consequence of Assumption 2. To show $\mathbb {E}[\delta _k \mid \mathcal {F}_k]=0$, we can write

$$\begin{aligned} \mathbb {E}[\delta _k \mid \mathcal {F}_k]&=\mathbb {E}\left[ \left[ {\rho _k} f_{j_k}(x_k)+\lambda _k^{(j_k)} \right] _+\tilde{\nabla }f_{j_k}(x_k)\right. \\&\left. \quad -\tfrac{1}{J}\sum _{j=1}^J \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\mid \mathcal {F}_k\right] \\&=\tfrac{1}{J}\sum _{j=1}^J \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\\&\quad -\tfrac{1}{J}\sum _{j=1}^J \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k) = 0, \end{aligned}$$

where the last inequality is implied from the assumption that $j_k$ is uniformly drawn from the set [J]. Next, we derive the bound on $\mathbb {E}[\Vert \delta _k\Vert ^2\mid \mathcal {F}_k]$. We have

$$\begin{aligned} \mathbb {E}[\Vert \delta _k\Vert ^2\mid \mathcal {F}_k]&= \mathbb {E}\left[ \left\| \left[ {\rho _k} f_{j_k}(x_k)+\lambda _k^{(j_k)} \right] _+\tilde{\nabla }f_{j_k}(x_k)\right\| ^2\mid \mathcal {F}_k\right] \\&\quad + \left\| \tfrac{1}{J}\sum _{j=1}^J \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\right\| ^2\\&\quad -2\mathbb {E}\left[ \left[ {\rho _k} f_{j_{k}}(x_k)+\lambda _k^{(j_{k})} \right] _+\tilde{\nabla }f_{j_{k}}(x_k)\mid \mathcal {F}_k\right] ^T\\&\quad \times \left( \tfrac{1}{J}\sum _{j=1}^J \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\right) \\&= \tfrac{1}{J}\sum _{j=1}^J\left\| \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\right\| ^2 \\&\quad - \left\| \tfrac{1}{J}\sum _{j=1}^J \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\right\| ^2. \end{aligned}$$

Dropping the non-negative term in the preceding relation and invoking Remark 5, we obtain

$$\begin{aligned} \mathbb {E}[\Vert \delta _k\Vert ^2\mid \mathcal {F}_k]&\le \tfrac{1}{J}\sum _{j=1}^J\left\| \left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+\tilde{\nabla }f_{j}(x_k)\right\| ^2 \\&= \tfrac{1}{J}\sum _{j=1}^J\left[ {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right] _+^2\left\| \tilde{\nabla }f_{j}(x_k)\right\| ^2\\&\le \tfrac{C_f^2}{J}\sum _{j=1}^J\left( {\rho _k} f_{j}(x_k)+\lambda _k^{(j)} \right) ^2\le \tfrac{2C_f^2}{J}\sum _{j=1}^J\left( \rho _k^2D_f^2+(\lambda _k^{(j)})^2\right) \\&= 2C_f^2\left( \rho _k^2D_f^2+\tfrac{\Vert \lambda _k\Vert ^2}{J}\right) . \end{aligned}$$

$\square$

1.4 Proof of Lemma 4

Proof

From the update rule of $v_{k+1}$, we know $\sigma _k={1\over \tau _k}(v_{k+1}-v_k)$, hence we have that

$$\begin{aligned} { \sigma ^T_{k}(x-v_k)}&={ \sigma ^T_{k}(x-v_{k+1})}+{ \sigma ^T_k (v_{k+1}-v_k)}\\&= {1\over 2\tau _k}\Vert x-v_k\Vert ^2-{1\over 2\tau _k}\Vert x-v_{k+1}\Vert ^2\\&\quad -{1\over 2\tau _k}\Vert v_{k+1}-v_k\Vert ^2+{ \sigma ^T_k(v_{k+1}-v_k)}\\&\le {1\over 2\tau _k}\Vert x-v_k\Vert ^2-{1\over 2\tau _k}\Vert x-v_{k+1}\Vert ^2+{\tau _k\over 2}\Vert \sigma _k\Vert ^2, \end{aligned}$$

where the first equality is obtained from three points equality in the Euclidean space and in the last inequality we used Young’s inequality. $\square$

1.5 Proof of Lemma 5

Proof

From the fact that $\lambda _{k+1}-\lambda _k=J\rho _k e_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)$, one can get the following:

$$\begin{aligned} \tfrac{1}{\rho _k}{ (\lambda _k-\lambda )^T (\lambda _{k+1}-\lambda _k)}&= { (\lambda _k-\lambda )^T (\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}\\&\quad + (\lambda _k-\lambda )^T (Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)-\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k)). \end{aligned}$$

Moreover, using the fact that $\tfrac{1}{\rho _k}{ (\lambda _k-\lambda )^T (\lambda _{k+1}-\lambda _k)} =\tfrac{1}{2\rho _k}\left( \Vert \lambda _{k+1}-\lambda \Vert ^2-\Vert \lambda _{k}-\lambda \Vert ^2-\Vert \lambda _{k+1}-\lambda _k\Vert ^2\right)$ and using previous equality one can obtain:

$$\begin{aligned} \tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda \Vert ^2&= \tfrac{1}{2\rho _k}\Vert \lambda _{k}-\lambda \Vert ^2+\tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda _k\Vert ^2+{ (\lambda _k-\lambda )^T (\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}\nonumber \\&\quad +{ (\lambda _k-\lambda )^T (Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)-\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}. \end{aligned}$$

(21)

Using (21), one can easily show that:

$$\begin{aligned}&-\Phi _{\rho _k}(x_k,\lambda _k)+{1\over J}\sum _{j=1}^J \lambda ^{(j)}f_j(x_k)+\tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda \Vert ^2 \nonumber \\&\quad =-\Phi _{\rho _k}(x_k,\lambda _k)+{1\over J}\sum _{j=1}^J \lambda ^{(j)}f_j(x_k)+ \tfrac{1}{2\rho _k}\Vert \lambda _{k}-\lambda \Vert ^2+\tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda _k\Vert ^2\nonumber \\&\qquad +{ (\lambda _k-\lambda )^T (\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}\nonumber \\&\qquad +{ (\lambda _k-\lambda )^T (Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)-\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}. \end{aligned}$$

(22)

From definition of $\Phi _{\rho _k}(x_k,\lambda _k)$, $J^+_k$ and $J_k^-$ we have:

$$\begin{aligned} \Phi _{\rho _k}(x_k,\lambda _k)=\tfrac{1}{J}\left[ \sum _{j\in J_k^+}(\tfrac{\rho _k}{2}(f_j(x_k))^2+\lambda _k^{(j)}f_j(x_k))-\sum _{j\in J_k^-}\tfrac{(\lambda _k^{(j)})^2}{2\rho _k}\right] . \end{aligned}$$

(23)

Using (22), (23) and the fact that $\nabla _{\lambda } \Phi _\rho (x,\lambda )=\tfrac{1}{J}\Big [\max (\tfrac{-\lambda ^{(j)}}{\rho },f_j(x))\Big ]_{j=1}^{J}$, the following holds:

$$\begin{aligned}&-\Phi _{\rho _k}(x_k,\lambda _k)+{1\over J}\sum _{j=1}^J \lambda ^{(j)}f_j(x_k)+\tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda \Vert ^2 \nonumber \\&\quad =-\tfrac{1}{J}\sum _{j\in J_k^+}\tfrac{\rho _k}{2}(f_j(x_k))^2+\tfrac{1}{J}\sum _{j\in J_k^-}\left[ \tfrac{(\lambda _k^{(j)})^2}{2\rho _k}+\lambda ^{(j)}f_j(x_k)+(\lambda _k^{(j)}-\lambda ^{(j)})\left( \tfrac{-\lambda _k^{(j)}}{\rho _k}\right) \right] \nonumber \\&\qquad +\tfrac{1}{2\rho _k}\Vert \lambda _{k}-\lambda \Vert ^2+\tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda _k\Vert ^2\nonumber \\&\qquad +{ (\lambda _k-\lambda )^T (\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}\nonumber \\&\qquad +{ (\lambda _k-\lambda )^T (Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)-\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}\nonumber \\&\quad = -\tfrac{1}{J}\sum _{j\in J_k^+}\tfrac{\rho _k}{2}(f_j(x_k))^2-\tfrac{1}{J}\sum _{j\in J_k^-}\left( \tfrac{(\lambda _k^{(j)})^2}{2\rho _k}-\lambda ^{(j)}\left( f_j(x_k)+\tfrac{\lambda _k^{(j)}}{\rho _k}\right) \right) \nonumber \\&\qquad +\tfrac{1}{2\rho _k}\Vert \lambda _{k}-\lambda \Vert ^2\nonumber \\&\qquad +\tfrac{1}{2\rho _k}\Vert \lambda _{k+1}-\lambda _k\Vert ^2+{ (\lambda _k-\lambda )^T (Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)-\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k))}. \end{aligned}$$

(24)

Note that $\lambda \ge 0$ and by definition $J_k^-$ it holds that $\lambda ^{(j)}(f_j(x_k)+\tfrac{\lambda _k^{(j)}}{\rho _k})\le 0$, so we conclude that

$$\begin{aligned}&-\tfrac{1}{J}\sum _{j\in J_k^+}\tfrac{\rho _k}{2}(f_j(x_k))^2-\tfrac{1}{J}\sum _{j\in J_k^-}\left( \tfrac{(\lambda _k^{(j)})^2}{2\rho _k}-\lambda ^{(j)}\left( f_j(x_k)+\tfrac{\lambda _k^{(j)}}{\rho _k}\right) \right) \nonumber \\&\quad \le -\tfrac{1}{J}\sum _{j\in J_k^+}\tfrac{\rho _k}{2}(f_j(x_k))^2-\tfrac{1}{J}\sum _{j\in J_k^-}\tfrac{(\lambda _k^{(j)})^2}{2\rho _k}. \end{aligned}$$

(25)

Hence we have the desired result by putting (25) in (24). $\square$

1.6 Proof of Lemma 6

Proof

(a) From definition of $\nabla _\lambda \Phi _{\rho _k}$, using Assumption 1(ii) and the fact that $\lambda _k^{(j_k)}\ge 0$ for all k and j, we have that $\Vert Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k)\Vert ^2=\left| \max \left( \tfrac{-\lambda _k^{(j_k)}}{{\rho _k}},f_{j_k}(x_k)\right) \right| ^2\le D_f^2.$

(b) By definition of $\bar{\sigma }_k$ and $\bar{v}_k$ and using Lemma 4, one can obtain the following.

$$\begin{aligned}&{ (\lambda -\lambda _k\pm \bar{v}_k)^T (\nabla _\lambda \Phi _{\rho _k}(x_k,\lambda _k)-Je_{j_k}\odot \nabla _{\lambda } \Phi _{\rho _k}(x_k,\lambda _k))}\\&\quad = { (\bar{v}_k-\lambda _k)^T \bar{\sigma }_k}+{ (\lambda -\bar{v}_k)^T \bar{\sigma }_k} \le {(\bar{v}_k-\lambda _k)^T \bar{\sigma }_k} +{1\over 2\bar{\tau }_k}\Vert \lambda -\bar{v}_k\Vert ^2\\\\&\qquad -{1\over 2\bar{\tau }_k}\Vert \lambda -\bar{v}_{k+1}\Vert +{\bar{\tau }_k\over 2}\Vert \bar{\sigma }_k\Vert ^2. \end{aligned}$$

$\square$