Improvement of Incomplete Multiview Clustering by the Tensor Reconstruction of the Connectivity Graph

Abstract

With the development of data collection technologies, a significant volume of multiview data has appeared, and their clustering has become topical. Most methods of multiview clustering assume that all views are fully observable. However, in many cases this is not the case. Several tensor methods have been proposed to deal with incomplete multiview data. However, the traditional tensor norm is computationally expensive, and such methods generally cannot handle undersampling and imbalances of various views. A new method for clustering incomplete multiview data is proposed. A new tensor norm is defined to reconstruct the connectivity graph, and the graphs are regularized to a consistent low-dimensional representation of patterns. The weights are then iteratively updated for each view. Compared to the existing ones, the proposed method not only determines the consistency between views but also obtains a low-dimensional representation of the samples using the resulting projection matrix. An efficient optimization algorithm based on the method of indefinite Lagrange multipliers is developed for the solution. The experimental results on four data sets demonstrate the effectiveness of the method.

APPENDIX

Proof of Theorems 1 and 2, as well as Lemma 1. Before proving Theorem 1, we consider Lemmas 2 and 3, which were presented in [55].

Lemma 2. We assume that for the optimization problem

$$\mathop {\min }\limits_{x > 0} \frac{1}{2}{{\left( {x - \sigma } \right)}^{2}} + w{{x}^{p}},$$

(A.1)

with p and $w$ given, the following threshold is defined

$${{\tau }_{p}}\left( w \right) = {{\left( {2w\left( {1 - p} \right)} \right)}^{{\frac{1}{{2 - p}}}}} + wp{{\left( {2w\left( {1 - p} \right)} \right)}^{{\frac{{p - 1}}{{2 - p}}}}}.$$

(A.2)

It can be concluded that the following points are valid:

(1) if $\sigma \leqslant {{\tau }_{p}}\left( w \right)$, then the optimal solution ${{x}_{p}}\left( {\sigma ,w} \right)$ from (A.1) is zero;

(2) if $\sigma > {{\tau }_{p}}\left( w \right)$, then the optimal solution ${{x}_{p}}\left( {\sigma ,w} \right) =$ ${\text{sign}}\left( \sigma \right){{S}_{p}}\left( {\sigma ,w} \right)$, where ${{S}_{p}}\left( {\sigma ,w} \right)$ can be obtained from ${{S}_{p}}\left( {\sigma ,w} \right) - \sigma + wp{{\left( {{{S}_{p}}\left( {\sigma ,w} \right)} \right)}^{{p - 1}}} = 0$.

Lemma 3. We assume that $Y = {{U}_{Y}}{{\Sigma }_{Y}}V_{Y}^{ \top }$ is a singular decomposition, $Y \in {{\mathbb{R}}^{{m \times n}}}$, $\tau > 0$, $l = \min (m,n)$, and ${{w}_{1}} \geqslant {{w}_{2}} \geqslant \; \cdots \; \geqslant {{w}_{l}} \geqslant 0$. Then the global optimum of the problem

$$\mathop {\min }\limits_X \quad \tau \left\| X \right\|_{{w,{{S}_{p}}}}^{p} + \frac{1}{2}\left\| {X - Y} \right\|_{F}^{2}$$

(A.3)

will be

$${{S}_{{\tau ,w,p}}}\left( Y \right) = {{U}_{Y}}{{P}_{{\tau ,w,p}}}\left( Y \right)V_{Y}^{ \top }\;,$$

(A.4)

where ${{P}_{{\tau ,w,p}}}\left( Y \right) = {\text{diag}}\left( {{{\gamma }_{1}},{{\gamma }_{2}},\; \ldots ,\;{{\gamma }_{l}}} \right)$ and ${{\gamma }_{i}} = {{x}_{p}}\left( {{{\sigma }_{i}}\left( Y \right),\tau {{w}_{i}}} \right)$ can be obtained according to Lemma 2.

Proof of Theorem 1. For the given semiorthogonal matrix $\Phi \in {{\mathbb{R}}^{{{{n}_{3}} \times r}}}$, there exists a semiorthogonal matrix ${{\Phi }^{c}} \in {{\mathbb{R}}^{{{{n}_{3}} \times \left( {{{n}_{3}} - r} \right)}}}$ satisfying ${{\overline \Phi }^{ \top }}\overline \Phi = I$, where $\bar {\Phi } = \left[ {\Phi ,{{\Phi }^{c}}} \right] \in {{\mathbb{R}}^{{{{n}_{3}} \times {{n}_{3}}}}}$. By definition, we have

$$\begin{gathered} \mathcal{X}\text{*} = \mathop {\arg \min }\limits_\mathcal{X} \tau \left\| \mathcal{X} \right\|_{{\Phi ,w,{{S}_{p}}}}^{p} + \frac{1}{2}\left\| {\mathcal{X} - \mathcal{A}} \right\|_{F}^{2} = \mathop {\arg \min }\limits_\mathcal{X} \tau \sum\limits_{i = 1}^r \left\| {X_{\Phi }^{{\left( i \right)}}} \right\|_{{w,{{S}_{p}}}}^{p} + \frac{1}{2}\left\| {{{\mathcal{X}}_{{\overline \Phi }}} - {{\mathcal{A}}_{{\overline \Phi }}}} \right\|_{F}^{2} \\ = \;\mathop {\arg \min }\limits_\mathcal{X} \tau \sum\limits_{i = 1}^r \left\| {X_{\Phi }^{{(i)}}} \right\|_{{w,{{S}_{p}}}}^{p} + \frac{1}{2}\sum\limits_{i = 1}^r \left\| {X_{\Phi }^{{(i)}} - A_{\Phi }^{{(i)}}} \right\|_{F}^{2} + \frac{1}{2}\sum\limits_{j = 1}^{{{n}_{3}} - r} \left\| {X_{{{{\Phi }^{c}}}}^{{(j)}} - A_{{{{\Phi }^{c}}}}^{{(j)}}} \right\|_{F}^{2}. \\ \end{gathered} $$

(A.5)

In (A.5) the variables $X_{\Phi }^{{(i)}}$ and $X_{{{{\Phi }^{c}}}}^{{(j)}}$ are independent. Thus, the task can be divided into ${{n}_{3}}$ independent subtasks:

$$\mathop {\min }\limits_{X_{\Phi }^{{(i)}}} \tau \left\| {X_{\Phi }^{{(i)}}} \right\|_{{w,{{S}_{p}}}}^{p} + \frac{1}{2}\left\| {X_{\Phi }^{{(i)}} - A_{\Phi }^{{(i)}}} \right\|_{F}^{2},\quad i = \overline {1,r} ,$$

(A.6)

and

$$\mathop {\min }\limits_{X_{{{{\Phi }^{c}}}}^{{(j)}}} \quad \frac{1}{2}\left\| {X_{{{{\Phi }^{c}}}}^{{(j)}} - A_{{{{\Phi }^{c}}}}^{{(j)}}} \right\|_{F}^{2},\quad j = \overline {1,{{n}_{3}} - r} .$$

(A.7)

According to Lemma 3, the optimal solutions (A.6) and (A.7) are $X{{_{\Phi }^{{(i)}}}^{*}} = {{U}_{{A_{\Phi }^{{(i)}}}}}{{P}_{{\tau ,w,p}}}\left( {A_{\Phi }^{{(i)}}} \right)V_{{A_{\Phi }^{{(i)}}}}^{ \top }$ and $X{{_{{{{\Phi }^{c}}}}^{{(j)}}}^{*}} = A_{{{{\Phi }^{c}}}}^{{(j)}}$, respectively. Thus, we obtain the optimal solutions

$$\begin{gathered} \mathcal{X}_{{\overline \Phi }}^{*} = {\text{fold}}\left( {X{{{_{\Phi }^{{(1)}}}}^{*}};X{{{_{\Phi }^{{(2)}}}}^{*}};\; \cdots ;\;X{{{_{\Phi }^{{(r)}}}}^{*}};X{{{_{{{{\Phi }^{c}}}}^{{(1)}}}}^{*}};X{{{_{{{{\Phi }^{c}}}}^{{(2)}}}}^{*}};\; \cdots ;\;X{{{_{{{{\Phi }^{c}}}}^{{({{n}_{3}} - r)}}}}^{*}}} \right) \\ = \;{\text{fold}}\left( {{{S}_{{\tau ,w,p}}}\left( {A_{\Phi }^{{(1)}}} \right);\; \cdots ;\;{{S}_{{\tau ,w,p}}}\left( {A_{\Phi }^{{(r)}}} \right);A_{{{{\Phi }^{c}}}}^{{(1)}};\; \cdots ;\;A_{{{{\Phi }^{c}}}}^{{({{n}_{3}} - r)}}} \right) \\ = \;{\text{fold}}\left( {{\text{unfold}}\left( {{{\mathcal{S}}_{{\tau ,w,p}}}\left( {{{\mathcal{A}}_{\Phi }}} \right)} \right);{\text{unfold}}\left( {{{\mathcal{A}}_{{{{\Phi }^{c}}}}}} \right)} \right)\;. \\ \end{gathered} $$

(A.8)

Further, the optimal solution to (2.32) is $\mathcal{X}{\text{*}} = {{\left( {\mathcal{X}_{{\overline \Phi }}^{*}} \right)}_{{{{{\overline \Phi }}^{ \top }}}}}$.

Proof of Lemma 1. From the limitation to $\mathcal{G}$ in (2.12), it is clear that $\left\{ {{{\mathcal{G}}^{k}}} \right\}$ should be limited. Further, according to (2.18), it is known that {A^k} and $\left\{ {\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m}} \right\}$ are also limited. In the kth iteration in Algorithm 2, we have

$$\begin{gathered} {{\left\| {{{\mathcal{C}}^{{k + 1}}}} \right\|}_{F}} = {{\left\| {{{\mathcal{C}}^{k}} + {{\rho }^{k}}\left( {{{\mathcal{G}}^{{k + 1}}} - {{\mathcal{Y}}^{{k + 1}}}} \right)} \right\|}_{F}} = {{\rho }^{k}}{{\left\| {{{\mathcal{G}}^{{k + 1}}} + \frac{{{{\mathcal{C}}^{k}}}}{{{{\rho }^{k}}}} - {{\mathcal{Y}}^{{k + 1}}}} \right\|}_{F}} \\ = \;{{\rho }^{k}}{{\left\| {{{{\left( {{{\mathcal{G}}^{{k + 1}}} + \frac{{{{\mathcal{C}}^{k}}}}{{{{\rho }^{k}}}} - {{\mathcal{Y}}^{{k + 1}}}} \right)}}_{{\overline \Phi }}}} \right\|}_{F}} = {{\rho }^{k}}{{\left\| {\mathcal{B}_{{\overline \Phi }}^{k} - \mathcal{Y}_{{\overline \Phi }}^{{k + 1}}} \right\|}_{F}}. \\ \end{gathered} $$

(A.9)

From (2.34) we get

$$\mathcal{Y}_{{\overline \Phi }}^{{k + 1}} = {\text{fold}}\left( {{\text{unfold}}\left( {{{\mathcal{S}}_{{\frac{\mu }{{{{\rho }^{k}}}},w,p}}}(\mathcal{B}_{\Phi }^{k})} \right);{\text{unfold}}\left( {\mathcal{B}_{{{{\Phi }^{c}}}}^{k}} \right)} \right).$$

(A.10)

Substituting (A.10) into (A.9), we have

$$\begin{gathered} {{\left\| {{{\mathcal{C}}^{{k + 1}}}} \right\|}_{F}} = {{\rho }^{k}}{{\left\| {\mathcal{B}_{{\overline \Phi }}^{k} - {\text{fold}}\left( {{\text{unfold}}\left( {{{\mathcal{S}}_{{\frac{\mu }{{{{\rho }^{k}}}},w,p}}}\left( {\mathcal{B}_{\Phi }^{k}} \right)} \right);\,\,{\text{unfold}}\left( {\mathcal{B}_{{{{\Phi }^{c}}}}^{k}} \right)} \right)} \right\|}_{F}} \\ = \;{{\rho }^{k}}{{\left\| {\mathcal{B}_{\Phi }^{k} - {{\mathcal{S}}_{{\frac{\mu }{{{{\rho }^{k}}}},w,p}}}\left( {\mathcal{B}_{\Phi }^{k}} \right)} \right\|}_{F}} \leqslant {{\rho }^{k}}\sqrt {\sum\limits_{i = 1}^r \sum\limits_{j = 1}^{{\text{min}}\left( {{{n}_{1}},{{n}_{2}}} \right)} {{{\left( {\frac{{J{{w}_{j}}\mu }}{{{{\rho }^{k}}}}} \right)}}^{2}}} = \sqrt {\sum\limits_{i = 1}^r \sum\limits_{j = 1}^{\min\left( {{{n}_{1}},{{n}_{2}}} \right)} {{{\left( {J{{w}_{j}}\mu } \right)}}^{2}}} , \\ \end{gathered} $$

(A.11)

where J is the number of iterations in solving problem (A.1). Thus, $\left\{ {{{\mathcal{C}}^{k}}} \right\}$ is bounded. Further, according to (2.34), we see that

$$\begin{gathered} {{\left\| {{{\mathcal{Y}}^{{k + 1}}}} \right\|}_{F}} = {{\left\| {{\text{fold}}\left( {{\text{unfold}}\left( {{{\mathcal{S}}_{{\frac{\mu }{\rho },w,p}}}(\mathcal{B}_{\Phi }^{k})} \right);\,\,{\text{unfold}}\left( {\mathcal{B}_{{{{\Phi }^{c}}}}^{k}} \right)} \right)} \right\|}_{F}} \\ \leqslant \;{{\left\| {{{\mathcal{S}}_{{\frac{\mu }{\rho },w,p}}}(\mathcal{B}_{\Phi }^{k})} \right\|}_{F}} + {{\left\| {\mathcal{B}_{{{{\Phi }^{c}}}}^{k}} \right\|}_{F}} \leqslant \sum\limits_{i = 1}^r {{\left\| {{{P}_{{\tau ,w,p}}}(\mathcal{B}{{{_{\Phi }^{k}}}^{{\left( i \right)}}})} \right\|}_{F}} + {{\left\| {\mathcal{B}_{{{{\Phi }^{c}}}}^{k}} \right\|}_{F}}. \\ \end{gathered} $$

(A.12)

It has already been mentioned that $\left\{ {{{\mathcal{C}}^{k}}} \right\}$ and $\left\{ {{{\mathcal{G}}^{k}}} \right\}$ are bounded; therefore, $\left\{ {{{\mathcal{B}}^{k}}} \right\}$ is bounded, which means $\left\{ {{{\mathcal{Y}}^{k}}} \right\}$ is also bounded.

Below we prove that the extended Lagrange function (2.16) is bounded. Note that we cannot assert that (2.16) is bounded based on the fact that all variables are bounded, since $\left\{ {{{\rho }^{k}}} \right\}$ is not bounded. Let us first prove that

$${{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{{k + 1}}},\left\{ {W_{i}^{{k + 1}}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{k}}} \right) \leqslant {{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{k}},\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{k}}} \right).$$

(A.13)

Recall (2.17), assume $f\left( {A,{{W}_{i}}} \right) = \left\| {\left( {A - {{W}_{i}}^{ \top }{{X}_{i}}} \right){{P}_{i}}} \right\|_{F}^{2} + \lambda {\text{tr}}\left( {A{{L}_{{G_{i}^{k}}}}{{A}^{ \top }}} \right)$ and, combined with $\delta _{i}^{k} = {{{{n}_{i}}} \mathord{\left/ {\vphantom {{{{n}_{i}}} {\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} }}} \right. \kern-0em} {\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} }}$, it can be deduced that

$$\sum\limits_{i = 1}^m {\frac{{{{n}_{i}}f\left( {{{A}^{{k + 1}}},W_{i}^{{k + 1}}} \right)}}{{2\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} }}} \leqslant \sum\limits_{i = 1}^m \frac{{{{n}_{i}}f\left( {{{A}^{k}},W_{i}^{k}} \right)}}{{2\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} }}.$$

(A.14)

For any positive numbers a and b, the following inequality holds:

$$a - \frac{{{{a}^{2}}}}{{2b}} \leqslant b - \frac{{{{b}^{2}}}}{{2b}}\;.$$

(A.15)

Moreover, since $f\left( {A,{{W}_{i}}} \right) \geqslant 0$, we have

$$\sum\limits_{i = 1}^m {{{n}_{i}}} \left( {\sqrt {f\left( {{{A}^{{k + 1}}},W_{i}^{{k + 1}}} \right)} - \frac{{f\left( {{{A}^{{k + 1}}},W_{i}^{{k + 1}}} \right)}}{{2\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} }}} \right) \leqslant \sum\limits_{i = 1}^m {{{n}_{i}}} \left( {\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} - \frac{{f\left( {{{A}^{k}},W_{i}^{k}} \right)}}{{2\sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} }}} \right).$$

(A.16)

Combining (A.14) and (A.16), we obtain

$$\sum\limits_{i = 1}^m {{{n}_{i}}} \sqrt {f\left( {{{A}^{{k + 1}}},W_{i}^{{k + 1}}} \right)} \leqslant \sum\limits_{i = 1}^m {{{n}_{i}}} \sqrt {f\left( {{{A}^{k}},W_{i}^{k}} \right)} .$$

(A.17)

Thus, Eq. (A.13) is derived. In Sections 2.2.2 and 2.2.3, since subtasks $\mathcal{G}$ and $\mathcal{Y}$ have an optimal solution, it is clear that

$$\begin{gathered} {{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{{k + 1}}},\left\{ {W_{i}^{{k + 1}}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{{k + 1}}},{{\mathcal{Y}}^{{k + 1}}},{{\mathcal{C}}^{k}}} \right) \leqslant \;{{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{{k + 1}}},\left\{ {W_{i}^{{k + 1}}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{k}}} \right) \\ \leqslant \;{{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{k}},\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{k}}} \right). \\ \end{gathered} $$

(A.18)

Then,

$$\begin{gathered} {{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{k}},\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{k}}} \right) = {{L}_{{{{\rho }^{{k - 1}}}}}}\left( {{{A}^{k}},\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{{k - 1}}}} \right) + \frac{{{{\rho }^{k}} - {{\rho }^{{k - 1}}}}}{2}\left\| {{{\mathcal{G}}^{k}} - {{\mathcal{Y}}^{k}}} \right\|_{F}^{2} \\ + \;\left\langle {{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}},{{\mathcal{G}}^{k}} - {{\mathcal{Y}}^{k}}} \right\rangle = {{L}_{{{{\rho }^{{k - 1}}}}}}\left( {{{A}^{k}},\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{{k - 1}}}} \right) \\ + \;\frac{{{{\rho }^{k}} - {{\rho }^{{k - 1}}}}}{2}\left\| {\frac{{{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}}}}{{{{\rho }^{{k - 1}}}}}} \right\|_{F}^{2} + \left\langle {{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}},\frac{{{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}}}}{{{{\rho }^{{k - 1}}}}}} \right\rangle \\ = \;{{L}_{{{{\rho }^{{k - 1}}}}}}\left( {{{A}^{k}},\left\{ {W_{i}^{k}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{k}},{{\mathcal{Y}}^{k}},{{\mathcal{C}}^{{k - 1}}}} \right) + \frac{{{{\rho }^{k}} + {{\rho }^{{k - 1}}}}}{{2{{{({{\rho }^{{k - 1}}})}}^{2}}}}\left\| {{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}}} \right\|_{F}^{2}. \\ \end{gathered} $$

(A.19)

Assume ${{b}_{c}}$ is the boundary $\left\| {{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}}} \right\|_{F}^{2}$. Combining Eqs. (A.18), we obtain

$${{L}_{{{{\rho }^{k}}}}}\left( {{{A}^{{k + 1}}},\left\{ {W_{i}^{{k + 1}}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{{k + 1}}},{{\mathcal{Y}}^{{k + 1}}},{{\mathcal{C}}^{k}}} \right) \leqslant {{L}_{{{{\rho }^{0}}}}}\left( {{{A}^{1}},\left\{ {W_{i}^{1}} \right\}_{{i = 1}}^{m},{{\mathcal{G}}^{1}},{{\mathcal{Y}}^{1}},{{\mathcal{C}}^{0}},} \right) + {{b}_{c}}\sum\limits_{k = 1}^\infty \frac{{{{\rho }^{k}} + {{\rho }^{{k - 1}}}}}{{2{{{({{\rho }^{{k - 1}}})}}^{2}}}}.$$

(A.20)

Since $\rho > 1$, the following inequality holds:

$$\sum\limits_{k = 1}^\infty {\frac{{{{\rho }^{k}} + {{\rho }^{{k - 1}}}}}{{2{{{({{\rho }^{{k - 1}}})}}^{2}}}}} \leqslant \sum\limits_{k = 1}^\infty {\frac{{{{\rho }^{k}}}}{{{{{({{\rho }^{{k - 1}}})}}^{2}}}}} = \eta \sum\limits_{k = 1}^\infty {\frac{1}{{{{\rho }^{{k - 1}}}}}} < + \infty .$$

(A.21)

Combining (A.19)–(A.21), the extended Lagrange function (2.16) is bounded.

Proof of Theorem 2. According to Lemma 1, we know that $\left\{ {{{\mathcal{C}}^{k}}} \right\}$, $\left\{ {{{\mathcal{G}}^{k}}} \right\}$, and $\left\{ {{{\mathcal{Y}}^{k}}} \right\}$ are all limited. The Bolzano–Weierstrass theorem states that every bounded sequence of real numbers has a convergent subsequence. Therefore, there is at least one accumulation point for $\left\{ {{{\mathcal{C}}^{k}}} \right\}$, $\left\{ {{{\mathcal{G}}^{k}}} \right\}$ and $\left\{ {{{\mathcal{Y}}^{k}}} \right\}$, respectively. In particular, we can get

$$\mathop {\lim }\limits_{k \to \infty } {{\left\| {{{\mathcal{G}}^{k}} - {{\mathcal{Y}}^{k}}} \right\|}_{F}} = \mathop {\lim }\limits_{k \to \infty } \frac{1}{{{{\rho }^{k}}}}{{\left\| {{{\mathcal{C}}^{k}} - {{\mathcal{C}}^{{k - 1}}}} \right\|}_{F}} = 0.$$

(A.22)

As in Section 2.2.2, each tube of ${{\mathcal{G}}^{k}}$ can be analyzed independently. If we remember that ${{g}^{{k + 1}}}$ is updated by method (2.27), then the following inequality holds:

$$\begin{gathered} {{\left\| {{{g}^{{k + 1}}} - {{g}^{k}}} \right\|}_{F}} = {{\left\| \begin{gathered} \frac{{{{\rho }^{k}}}}{{{{\rho }^{k}} + \gamma }}{\text{max}}\left( {{{P}_{w}}\left( {{{u}^{k}} - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}{{u}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}}} \right),0} \right) \\ + \,{\text{max}}\left( {{{P}_{{{{w}^{c}}}}}\left( {{{u}^{k}} - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}{{u}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}}} \right),0} \right) - {{g}^{k}} \\ \end{gathered} \right\|}_{F}} \\ \leqslant {{\left\| {\frac{{{{\rho }^{k}}}}{{{{\rho }^{k}} + \gamma }}\left( {{{u}^{k}} - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}{{u}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}} - \frac{{{{\rho }^{k}} + \gamma }}{{{{\rho }^{k}}}}{{g}^{k}}} \right)} \right\|}_{F}} + {{\left\| {{{u}^{k}} - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}{{u}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}} - {{g}^{k}}} \right\|}_{F}}. \\ \end{gathered} $$

(A.23)

Let us analyze the first and second terms (A.23) separately:

$$\begin{gathered} {{\left\| {\frac{{{{\rho }^{k}}}}{{{{\rho }^{k}} + \gamma }}\left( {{{u}^{k}} - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}{{u}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}} - \frac{{{{\rho }^{k}} + \gamma }}{{{{\rho }^{k}}}}{{g}^{k}}} \right)} \right\|}_{F}} \\ = \;{{\left\| {\frac{{{{\rho }^{k}}}}{{{{\rho }^{k}} + \gamma }}\left( \begin{gathered} {{y}^{k}} - \frac{{{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}} \\ - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}\left( {{{y}^{k}} - \frac{{{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}}} \right) \\ + \frac{1}{n}{\mathbf{1}} - {{g}^{k}} - \frac{\gamma }{{{{\rho }^{k}}}}{{g}^{k}} + {v}1 \\ \end{gathered} \right)} \right\|}_{F}} \\ = \;{{\left\| {\frac{{{{\rho }^{k}}}}{{{{\rho }^{k}} + \gamma }}\left( \begin{gathered} \frac{{{{c}^{{k - 1}}} - 2{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}} \\ - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}\left( {\frac{{{{c}^{{k - 1}}} - 2{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}}} \right) \\ - \frac{\gamma }{{{{\rho }^{k}}}}{{g}^{k}} + {v}{\mathbf{1}} \\ \end{gathered} \right)} \right\|}_{F}} \\ \end{gathered} $$

(A.24)

and

$$\begin{gathered} {{\left\| {{{u}^{k}} - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}{{u}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}} - {{g}^{k}}} \right\|}_{F}} \\ = \;{{\left\| \begin{gathered} {{y}^{k}} - \frac{{{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}} + \frac{1}{n}{\mathbf{1}} + {v}{\mathbf{1}} - {{g}^{k}} \\ - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}\left( {{{y}^{k}} - \frac{{{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}}} \right) \\ \end{gathered} \right\|}_{F}} \\ = \;{{\left\| \begin{gathered} \frac{{{{c}^{{k - 1}}} - {{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}} + {v}{\mathbf{1}} \\ - \frac{1}{n}{\mathbf{1}}{{{\mathbf{1}}}^{ \top }}\left( {\frac{{{{c}^{{k - 1}}} - {{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}}} \right) \\ \end{gathered} \right\|}_{F}}. \\ \end{gathered} $$

(A.25)

Below, we show that ${v}$ tends to 0 when k tends to infinity. According to the definition u^k, the following equation holds:

$$\mathop {\lim }\limits_{k \to \infty } {{\left\| {{{u}^{k}} - {{g}^{k}}} \right\|}_{2}} = \mathop {\lim }\limits_{k \to \infty } {{\left\| {{{g}^{k}} + \frac{{{{c}^{{k - 1}}} - 2{{c}^{k}}}}{{{{\rho }^{k}}}} + \frac{\gamma }{{{{\rho }^{k}}}}{{P}_{w}}\left( m \right) - \frac{1}{{{{\rho }^{k}}}}{{t}^{k}} - {{g}^{k}}} \right\|}_{2}} = 0.$$

(A.26)

Comparing (2.27) and (2.28), (A.26) implies $f\left( 0 \right) = 0$ when $k$ tends to infinity; i.e., ${v}$ tends to 0. Eventually, $\mathop {\lim }\limits_{k \to \infty } {{\left\| {{{g}^{{k + 1}}} - {{g}^{k}}} \right\|}_{2}} = 0$ and hence

$$\mathop {\lim }\limits_{k \to \infty } {{\left\| {{{\mathcal{G}}^{{k + 1}}} - {{\mathcal{G}}^{k}}} \right\|}_{F}} = 0.$$

(A.27)

For ${{\mathcal{Y}}^{k}}$, the following inequality is valid:

$$\begin{gathered} {{\left\| {{{\mathcal{Y}}^{{k + 1}}} - {{\mathcal{Y}}^{k}}} \right\|}_{F}} = {{\left\| {{{\mathcal{Y}}^{{k + 1}}} - {{\mathcal{G}}^{{k + 1}}} - {{\mathcal{Y}}^{k}} + {{\mathcal{G}}^{k}} + {{\mathcal{G}}^{{k + 1}}} - {{\mathcal{G}}^{k}}} \right\|}_{F}} \\ \leqslant {{\left\| {{{\mathcal{Y}}^{{k + 1}}} - {{\mathcal{G}}^{{k + 1}}}} \right\|}_{F}} + {{\left\| {{{\mathcal{Y}}^{k}} - {{\mathcal{G}}^{k}}} \right\|}_{F}} + {{\left\| {{{\mathcal{G}}^{{k + 1}}} - {{\mathcal{G}}^{k}}} \right\|}_{F}}. \\ \end{gathered} $$

(A.28)

Thus, combining (A.22) and (A.27), we have

$$\mathop {\lim }\limits_{k \to \infty } {{\left\| {{{\mathcal{Y}}^{{k + 1}}} - {{\mathcal{Y}}^{k}}} \right\|}_{F}} = 0.$$

(A.29)