k-Sparse Vector Recovery via $\ell _1-\alpha \ell _2$ Local Minimization

Abstract

This paper studies the $\ell _1-\alpha \ell _2$ local minimization model for $\alpha \in (0,2]$, which is the first time to consider the case of $\alpha >1$. We obtain the necessary and sufficient conditions for a fixed sparse signal to be recovered from this model. Based on this condition, we also obtain the necessary and sufficient conditions for any k-sparse signal to be recovered from $\ell _1-\alpha \ell _2$ local minimization model with $0<\alpha <1$, $\alpha =1$ and $1<\alpha \le 2$. The experimental data show that the size of $\alpha$ is positively correlated with the success rate of signal recovery.

Appendix A

1.1 Proof of Theorem 1

Proof

Necessity. From (7) and Remark 1,

$$\begin{aligned} f(x,th)=t\left( \sum \limits _{i\in S} \textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle -\alpha g(t)\right) >0. \end{aligned}$$

Hence, from (8)

$$\begin{aligned} \sum \limits _{i\in S} \textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle >\alpha g(t)\ge 0. \end{aligned}$$

Sufficiency. Let (9) holds for all $h\in \textrm{ker} A$ and $\Vert h\Vert _2=1$. Then there exists an $h^*\in \textrm{ker} A$ and $\Vert h^*\Vert _2=1$ such that

$$\begin{aligned} \begin{aligned}&\textrm{min}\left\{ \sum \limits _{i\in S} \textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle \right\} \\&=\sum \limits _{i\in S} \textrm{sign}(x_i)h^*_i+\Vert h^*_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h^*\right\rangle \\&=:\rho >0. \end{aligned} \end{aligned}$$

On the other hand,

$$\begin{aligned} 0\le g(t)= & {} \frac{t+2\langle x,h\rangle }{\Vert x+th\Vert _2+\Vert x\Vert _2}-\left\langle \frac{x}{\Vert x\Vert _2}, h\right\rangle \\= & {} \frac{(t+2\langle x,h\rangle )\Vert x\Vert _2-\langle x,h\rangle (\Vert x+th\Vert _2+\Vert x\Vert _2)}{(\Vert x+th\Vert _2+\Vert x\Vert _2)\Vert x\Vert _2}\\= & {} \frac{t\Vert x\Vert _2+\langle x, h\rangle \Vert x\Vert _2-\langle x,h\rangle \Vert x+th\Vert _2}{\Vert x\Vert _2(\Vert x+th\Vert _2+\Vert x\Vert _2)}\\{} & {} \le \frac{t\Vert x\Vert _2+|\langle x, h\rangle ||\Vert x\Vert _2-\Vert x+th\Vert _2|}{\Vert x\Vert _2(\Vert x+th\Vert _2+\Vert x\Vert _2)}\\{} & {} \le \frac{t\Vert x\Vert _2+|\langle x, h\rangle |\Vert th\Vert _2}{\Vert x\Vert _2(\Vert x+th\Vert _2+\Vert x\Vert _2)} \le \frac{2t\Vert x\Vert _2}{\Vert x\Vert _2(\Vert x+th\Vert _2+\Vert x\Vert _2)}\\{} & {} \le \frac{2t\Vert x\Vert _2}{\Vert x\Vert _2^2}=\frac{2t}{\Vert x\Vert _2}. \end{aligned}$$

Hence,

$$\begin{aligned} \begin{aligned} f(x,th)&=t\left( \sum \limits _{i\in S} \textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle -\alpha g(t)\right) \\&\ge t(\rho -\alpha g(t))\ge t\left( \rho - \alpha \frac{2t}{\Vert x\Vert _2}\right) >0, \end{aligned} \end{aligned}$$

when $t<\textrm{min}\left\{ \frac{\rho \Vert x\Vert _2}{2\alpha },\textrm{min}_{i\in S}|x_i|\right\}$. $\square$

1.2 Proof of Lemma 1

Proof

$(i)\Rightarrow (ii)$. $\forall h \in \textrm{ker} A$, $\Vert h\Vert _2=1$, and $\forall x\ne 0$, $\textrm{supp}(x)=S$, $|S|\le k$. Set

$$\begin{aligned} x^*_i= {\left\{ \begin{array}{ll} -\textrm{sign}(h_i)|x_i|&{} i\in S_1(x) \text { and }h_i\ne 0\\ \textrm{sign}(h_i)|x_i|&{} i\in S_2(x) \text { and }h_i\ne 0\\ x_i &{} i\in S \text { and }h_i=0\\ 0 &{} i\notin S \end{array}\right. }, \end{aligned}$$

then $\textrm{supp}(x^*)=S$ and

$$\begin{aligned} \begin{aligned}&\Vert h_{\overline{S}}\Vert _1-\sum \limits _{i\in S_1(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|-\sum \limits _{i\in S_2(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|\\&=\Vert h_{\overline{S}}\Vert _1+\sum \limits _{i\in S}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) \textrm{sign}(x^*_i)h_i\\&=\sum \limits _{i\in S}\textrm{sign}(x^*_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x^*}{\Vert x^*\Vert _2},\alpha h\right\rangle \\&>0. \end{aligned} \end{aligned}$$

$(ii)\Rightarrow (i)$. Note that $\forall h \in \textrm{ker} A$, $\Vert h\Vert _2=1$, and $\forall x\ne 0$, $\textrm{supp}(x)=S$, $|S|\le k$

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle \\&=\Vert h_{\overline{S}}\Vert _1+\sum \limits _{i\in S}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) \textrm{sign}(x_i)h_i\\&\ge \Vert h_{\overline{S}}\Vert _1-\sum \limits _{i\in S_1(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|-\sum \limits _{i\in S_2(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|\\&>0. \end{aligned} \nonumber \\ \end{aligned}$$

$\square$

1.3 Proof of Lemma 2

Proof

We note that, $\forall x\ne 0$, satisfying $\textrm{supp}(x)=S$, $S_1(x)\ne \emptyset$, $S_2(x)\ne \emptyset$, then for any $i\in S_1(x)$, we have

$$\begin{aligned} \begin{aligned} \frac{|x^*(x)_i|}{\Vert x^*(x)\Vert _2}&=\frac{t|x_i|}{\sqrt{\sum \limits _{j\in S_1(x)\cup S_2(x)}|x^*(x)_j|^2}}=\frac{t|x_i|}{\sqrt{\sum \limits _{j\in S_1(x)}t^2x_j^2+\sum \limits _{j\in S_2(x)}|x_i|^2}}\\&=\frac{|x_i|}{\sqrt{\sum \limits _{j\in S_1(x)}x_j^2+\sum \limits _{j\in S_2(x)}\frac{|x_i|^2}{t^2}}}<\frac{|x_i|}{\Vert x\Vert _2}, \end{aligned} \end{aligned}$$

which means that when all elements in $S_1(x)$ are reduced in absolute for the same proportion, any $i\in S_1(x)$, $\frac{|x_i|}{\Vert x\Vert _2}$ will also become smaller. In addition, for any $j\in S_2(x)$, $i\in S_1(x)$ it is easy to know $\frac{|x_j|}{\Vert x\Vert _2}$ increases as $|x_i|$ decreases. Hence, $S_1(x^*(x))=S_1(x)$, $S_2(x^*(x))=S_2(x)$ and

$$\begin{aligned}{} & {} \sum \limits _{i\in S_1(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|\le \sum \limits _{i\in S_1(x^*)}\left( 1-\frac{\alpha |x^*(x)_i|}{\Vert x^*(x)\Vert _2}\right) |h_i|,\\{} & {} \sum \limits _{i\in S_2(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|\le \sum \limits _{i\in S_2(x^*)}\left( \frac{\alpha |x^*(x)_i|}{\Vert x^*\Vert _2}-1\right) |h_i|. \end{aligned}$$

It follows that $f_{S,h}(x)\ge f_{S,h}(x^*(x))$. Similarly,

$$\begin{aligned}{} & {} \sum \limits _{i\in S_1(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|\le \sum \limits _{i\in S_1(x^*)}\left( 1-\frac{\alpha |x^{**}(x)_i|}{\Vert x^{**}(x)\Vert _2}\right) |h_i|,\\{} & {} \sum \limits _{i\in S_2(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|\le \sum \limits _{i\in S_2(x^*)}\left( \frac{\alpha |x^{**}(x)_i|}{\Vert x^{**}\Vert _2}-1\right) |h_i|. \end{aligned}$$

This yields, $f_{S,h}(x)\ge f_{S,h}(x^{**}(x))$. $\square$

1.4 Proof of Lemma 3

Proof

Set $j_0=\textrm{argmin}_{i\in S}|x_i|$. We gradually reduce $|x_{j_0}|$ but not to 0 and keep the other components in x unchanged. Set the changed vector as $x^*$. It is easy to know that $0<\frac{|x_i|}{\Vert x\Vert _2}-\frac{1}{\alpha }\le \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }$ for $i\in S\backslash \{j_0\}$. Additionally, because $1<\alpha \le 2$,

$$\begin{aligned} 0<\frac{|x_{j_0}|}{\Vert x\Vert _2}-\frac{1}{\alpha }=\min \limits _{i\in S}\frac{|x_i|}{\Vert x\Vert _2}-\frac{1}{\alpha }\le \frac{1}{\sqrt{S}}-\frac{1}{\alpha }\le \frac{1}{\sqrt{2}}-\frac{1}{\alpha }< \frac{1}{2}. \end{aligned}$$

However, when $|x^*_{j_0}|\rightarrow 0$, we have $\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\rightarrow 0$. Hence, $0\le \frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\rightarrow \frac{1}{\alpha }$. Since $\frac{1}{\alpha }\in [\frac{1}{2},1)$ we have $\frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\rightarrow \frac{1}{\alpha }\ge \frac{1}{2}>\frac{|x_{j_0}|}{\Vert x\Vert _2}-\frac{1}{\alpha }$ when $|x^*_{j_0}|\rightarrow 0$. Combined with the above, the following formula is generated,

$$\begin{aligned} \begin{aligned}&g_{S,h}(x)-g_{S,h}(x^*)\\&=\sum \limits _{i\in S}\left( \frac{|x_i|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_i|-\sum \limits _{i\i S_1(x^*)}\left( \frac{1}{\alpha }-\frac{|x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|-\sum \limits _{i\in S_2(x^*)}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_i|\\&=\sum \limits _{i\in S}\left( \frac{|x_i|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_i|-\left( \frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\right) |h_{j_0}|-\sum \limits _{i\in S\backslash \{j_0\}}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_i|\\&=\sum \limits _{i\in S\backslash \{j_0\}}\left( \frac{|x_i|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_i|+\left( \frac{|x_{j_0}|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_{j_0}|-\left( \frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\right) |h_{j_0}|\\&\quad -\sum \limits _{i\in S\backslash \{j_0\}}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_i|\le 0. \end{aligned} \end{aligned}$$

(16)

Therefore, the conclusion holds. $\square$

1.5 Proof of Lemma 4

Proof

Set $j_0=\textrm{argmin}_{i\in S_2(x)}|x_i|$, $j_1=\textrm{argmax}_{i\in S_2(x)}|x_i|$. Since $j_0\in S_2(x)$, $\frac{1}{\alpha }\in [\frac{1}{2},1)$, we have

$$\begin{aligned} 0<\frac{|x_{j_0}|}{\Vert x\Vert _2}-\frac{1}{\alpha }<\frac{|x_{j_0}|}{\Vert x_{S_2(x)}\Vert _2}-\frac{1}{\alpha }\le \frac{1}{\sqrt{|S_2(x)|}}-\frac{1}{\alpha }\le \frac{1}{\sqrt{2}}-\frac{1}{\alpha }<\frac{1}{2}\le \frac{1}{\alpha }. \end{aligned}$$

Now we reduce $|x_{j_0}|$ but not to 0, increase $|x_{j_1}|$, and keep the other components in x and $\Vert x\Vert _2$ constant. We set the changed vector as $x^*$. Obviously, $\frac{|x_{j_1}|}{\Vert x\Vert _2}<\frac{|x^*_{j_1}|}{\Vert x^*\Vert _2}$. In addition, when $|x^*_{j_0}|\rightarrow 0$, we have $\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\rightarrow 0$, and further more $0<\frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\rightarrow \frac{1}{\alpha }$. So when $|x^*_{j_0}|\rightarrow 0$, we have $\frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}>\frac{|x_{j_0}|}{\Vert x\Vert _2}-\frac{1}{\alpha }$. Combined with the above, the following formula is generated,

$$\begin{aligned} \begin{aligned}&g_{S,h}(x)-g_{S,h}(x^*)\\&=\sum \limits _{i\in S_1(x)}\left( \frac{1}{\alpha }-\frac{|x_i|}{\Vert x\Vert _2}\right) |h_i|+\sum \limits _{i\in S_2(x)}\left( \frac{|x_i|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_i|\\&\quad -\sum \limits _{i\in S_1(x^*)}\left( \frac{1}{\alpha }-\frac{|x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|-\sum \limits _{i\in S_2(x^*)}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_i|\\&=\left( \frac{|x_{j_0}|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_{j_0}|+\left( \frac{|x_{j_1}|}{\Vert x\Vert _2}-\frac{1}{\alpha }\right) |h_{j_1}|\\&\quad -\left( \frac{1}{\alpha }-\frac{|x^*_{j_0}|}{\Vert x^*\Vert _2}\right) |h_{j_0}|-\left( \frac{|x^*_{j_1}|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_{j_1}| \le 0. \end{aligned} \end{aligned}$$

(17)

Now we find an $x^*$ satisfying $\textrm{supp}(x^*)=\textrm{supp}(x)$, $g_{S,h}(x^*)\ge g_{S,h}(x)$ and $|S_2(x^*)|=|S_2(x)|-1$. If $|S_2(x^*)|\ge 2$, we continue the above operation and find an $x^{**}$ again such that $x^{**}$ satisfies $\textrm{supp}(x^{**})=\textrm{supp}(x^*)=\textrm{supp}(x)$, $g_{T,h}(x^{**})\ge g_{S,h}(x^*)\ge g_{S,h}(x)$ and $|S_2(x^{**})|=|S_2(x^*)|-1=|S_2(x)|-2$. Finally, we find an $\overline{x}$ satisfying $\textrm{supp}(\overline{x})=\textrm{supp}(x)$, $g_{S,h}(\overline{x})\ge g_{S,h}(x)$ and $|S_2(\overline{x})|=1$. $\square$

1.6 Proof of Lemma 5

Proof

We prove it in two cases.

(i) $0<\alpha \le 1$. First, $S_1\subset S_2$ implies $\Vert h_{\overline{S_1}}\Vert _1\ge \Vert h_{\overline{S_2}}\Vert _1$. Second, take

$$\begin{aligned} x^*_i= {\left\{ \begin{array}{ll} x_i &{} i\in S_1\\ 1&{} i\in S_2\backslash S_1\\ 0 &{} i\notin S_2 \end{array}\right. }, \end{aligned}$$

then $\textrm{supp}(x^*)=S_2$. In addition, since $0<\alpha \le 1$, we have $S_{12}(x)=\emptyset$, $S_{11}(x)=S_1$, $S_{22}(x^*)=\emptyset$, $S_{21}(x^*)=S_2$ where $S_{11}(x)$, $S_{12}(x)$, $S_{21}(x^*)$, $S_{22}(x^*)$ are defined as (10). So

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S_{11}(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|+\sum \limits _{i\in S_{12}(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|=\sum \limits _{i\in S_{1}}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|\\&\le \sum \limits _{i \in S_{1}}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|\le \sum \limits _{i \in S_{1}}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|+\sum \limits _{i\in S_2\backslash S_1}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|\\&=\sum \limits _{i \in S_{2}}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|=\sum \limits _{i\in S_{21}(x^*)}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|+\sum \limits _{i\in S_{22}(x^*)}\left( \frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}-1\right) |h_i|. \end{aligned} \end{aligned}$$

Combined with the above, it is easy to know $f_{S_2,h}(x^*)\le f_{S_1,h}(x)$.

(ii) $1<\alpha \le 2$. When $S_{12}(x)=\emptyset$, it is similar to case (i), the result is true. When $S_{12}(x)\ne \emptyset$, since $|S_1|>1$, Lemma 4 shows that there exists a vector $\overline{x}$ such that $\textrm{supp}(\overline{x})=S_1$, $S_{11}(\overline{x})\ne \emptyset$, $|S_{12}(\overline{x})|=1$, $f_{S_1,h}(\overline{x})\le f_{S_1,h}(x)$. Taking $l\in S_2\backslash S_1$ and setting $j_0\in S_{12}(\overline{x})$, we set vector $\tilde{x}$ as follows:

$$\begin{aligned} \tilde{x}_i= {\left\{ \begin{array}{ll} \overline{x}_i &{} i\in S_{11}(\overline{x})\\ 1 &{} i=l\\ \lambda \overline{x}_i &{} i=j_0 \end{array}\right. }, \end{aligned}$$

where $\lambda \ge 1$ is a constant such that $\frac{\tilde{x}_{j_0}}{\Vert \tilde{x}\Vert _2}=\frac{\overline{x}_{j_0}}{\Vert \overline{x}\Vert _2}$. This implies that $\textrm{supp}(\tilde{x})=S_1\cup \{l\}$. It follows that

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S_{11}(\overline{x})}\left( 1-\frac{\alpha |\overline{x}_i|}{\Vert \overline{x}\Vert _2}\right) |h_i|+\sum \limits _{i\in S_{12}(\overline{x})}\left( \frac{\alpha |\overline{x}_i|}{\Vert \overline{x}\Vert _2}-1\right) |h_i|\\&\quad =\sum \limits _{i\in S_{1}\backslash j_0}\left( 1-\frac{\alpha |\overline{x}_i|}{\Vert \overline{x}\Vert _2}\right) |h_i|+\left( \frac{\alpha |\overline{x}_{j_0}|}{\Vert \overline{x}\Vert _2}-1\right) |h_{j_0}|\\&\quad \le \sum \limits _{i\in S_{1}\backslash j_0}\left( 1-\frac{\alpha |\tilde{x}_i|}{\Vert \tilde{x}\Vert _2}\right) |h_i|+\left( \frac{\alpha |\tilde{x}_{j_0}|}{\Vert \tilde{x}\Vert _2}-1\right) |h_{j_0}|\\&\quad \le \sum \limits _{i\in S_{1}\backslash j_0}\left( 1-\frac{\alpha |\tilde{x}_i|}{\Vert \tilde{x}\Vert _2}\right) |h_i|+\left( \frac{\alpha |\tilde{x}_{j_0}|}{\Vert \tilde{x}\Vert _2}-1\right) |h_{j_0}|+\left| \frac{\alpha |\tilde{x}_{l}|}{\Vert \tilde{x}\Vert _2}-1\right| |h_l|\\&\quad =\sum \limits _{i\in S_{11}(\tilde{x})}\left( 1-\frac{\alpha |\tilde{x}_i|}{\Vert \tilde{x}\Vert _2}\right) |h_i|+\sum \limits _{i\in S_{12}(\tilde{x})}\left( \frac{\alpha |\tilde{x}_i|}{\Vert \tilde{x}\Vert _2}-1\right) |h_i|, \end{aligned} \end{aligned}$$

which means $g_{S_1\cup \{l\},h}(\tilde{x})\ge g_{S_1,h}(\overline{x})$, i.e., $f_{S_1\cup \{l\},h}(\tilde{x})\le f_{S_1,h}(\overline{x})$. If $S_1\cup \{l\}=S_2$, we stop, and find a vector $x^*=\tilde{x}$ satisfying $\textrm{supp}(x^*)=S_2$, $f_{S_2,h}(x^*)\le f_{S_1,h}(x)$. If not, we continue to repeat the previous operation and eventually find a vector $x^*$ satisfying $\textrm{supp}(x^*)=S_2$, $f_{S_2,h}(x^*)\le f_{S_1,h}(x)$. $\square$

1.7 Proof of Lemma 6

Proof

First, the choice of T indicates that $\Vert h_{\overline{T}}\Vert _1\le \Vert h_{\overline{S}}\Vert _1$. Second, $\forall x$, satisfying $\textrm{supp}(x)=S$, take $x^*$, satisfying $\textrm{supp}(x^*)=T$, and $x^*_{\sigma _i(h_T)}=x_{\sigma _i(h_S)}$, where $\sigma _i(h)$ means the subscript of the $i-$th largest component of |h|. From the choice method of $x^*$, we have

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S_1(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|+\sum \limits _{i\in S_2(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|\\&\quad \le \sum \limits _{i \in T_1(x^*)}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|+\sum \limits _{i\in T_2(x^*)}\left( \frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}-1\right) |h_i|, \end{aligned} \end{aligned}$$

where $S_1(x)$, $S_2(x)$, $T_1(x^*)$, $T_2(x^*)$ are defined in (10). Combined with the above, it is easy to determine that $f_{T,h}(x^*)\le f_{S,h}(x)$. $\square$

1.8 Proof of Lemma 7

Proof

$(i)\Leftrightarrow (ii)$ is Lemma 1.

$(ii)\Rightarrow (iii)$. Taking $S=T$ in (ii), then (iii) holds obviously.

$(iii)\Rightarrow (ii)$. We discuss this in terms of two cases.

$(*)$. $|S|>1$. Lemmas 5 and 6 show that for an arbitrary subscript set S satisfying $1<|S|\le k$, and arbitrary k-sparse vector x, $\textrm{supp}(x)=S$, there exists an $x^*$ satisfying $\textrm{supp}(x^*)=T$ such that $f_{S,h}(x)\ge f_{T,h}(x^*)$ holds. Hence, if (iii) holds, then (ii) holds too.

$(**)$ $|S|=1$. If $0<\alpha \le 1$, from Remark 6, Lemma 5 holds; hence, similar to the proof of case $(*)$, the result holds. Now, we assume that $1<\alpha \le 2$. In this case, $S_{2}(x)=S:=\{l\}$. and

$$\begin{aligned} f_{S,h}(x)= & {} \Vert h_{\overline{S}}\Vert _1-\left( \frac{\alpha |x_l|}{\Vert x\Vert _2}-1\right) |h_l|=\Vert h_{\overline{S}}\Vert _1-(\alpha -1)|h_l|=\Vert h\Vert _1-\alpha |h_l|\\{} & {} \ge \Vert h\Vert _1-\alpha |h_{j_0}|, \end{aligned}$$

where $j_0=\textrm{argmax}_i\{|h_i|\}$. Setting $j_1=\textrm{argmax}_i\{|h_i|,i\ne j_0\}$, we assume that $|h_{j_1}|\ne 0$, which is normal, because $|h_{j_1}|=0$ if and only if A contains a column of zero vectors, such A is not allowed. We define the vector $x^*$ as follows:

$$\begin{aligned} \begin{aligned} x^*_i= {\left\{ \begin{array}{ll} x_l &{} i=j_0\\ \lambda x^l&{} i=j_1\\ 0 &{} \textrm{other} \end{array}\right. }, \end{aligned} \end{aligned}$$

(18)

where $\lambda$ is a sufficiently small number such that $\frac{\lambda \alpha }{\sqrt{1+\lambda ^2}}\le 1\le \frac{\alpha }{\sqrt{1+\lambda ^2}}$. Setting $\overline{S}=\{j_0,j_1\}$, then $\textrm{supp}(x^*)=\overline{S}$, $|\overline{S}|>1$ and

$$\begin{aligned} \begin{aligned} f_{\overline{S},h}(x^*)&=\Vert h_{\overline{(\overline{S})}}\Vert _1-\left( 1-\frac{\alpha |x^*_{j_1}|}{\Vert x^*\Vert _2}\right) |h_{j_1}|-\left( \frac{\alpha |x^*_{j_0}|}{\Vert x^*\Vert _2}-1\right) |h_{j_0}|\\&=\Vert h\Vert _1-\left( 2-\frac{\alpha |x^*_{j_1}|}{\Vert x^*\Vert _2}\right) |h_{j_1}|-\frac{\alpha |x^*_{j_0}|}{\Vert x^*\Vert _2}|h_{j_0}|\\&=\Vert h\Vert _1-\alpha |h_{j_0}|+\alpha |h_{j_0}|-\left( 2-\frac{\alpha |x^*_{j_1}|}{\Vert x^*\Vert _2}\right) |h_{j_1}|-\frac{\alpha |x^*_{j_0}|}{\Vert x^*\Vert _2}|h_{j_0}|\\&=\Vert h\Vert _1-\alpha |h_{j_0}|-\left( 2-\frac{\alpha |x^*_{j_1}|}{\Vert x^*\Vert _2}\right) |h_{j_1}|-\left( \frac{\alpha |x^*_{j_0}|}{\Vert x^*\Vert _2}-\alpha \right) |h_{j_0}|\\&=\Vert h\Vert _1-\alpha |h_{j_0}|-\left( 2-\frac{\alpha \lambda }{\sqrt{1+\lambda ^2}}\right) |h_{j_1}|-\left( \frac{\alpha }{\sqrt{1+\lambda ^2}}-\alpha \right) |h_{j_0}|. \end{aligned} \end{aligned}$$

Since

$$\begin{aligned} \lim \limits _{\lambda \longrightarrow 0^+}\left( 2-\frac{\alpha \lambda }{\sqrt{1+\lambda ^2}}\right) |h_{j_1}|+\left( \frac{\alpha }{\sqrt{1+\lambda ^2}}-\alpha \right) |h_{j_0}|=2|h_{j_1}|>0, \end{aligned}$$

there is r such that for $0<\lambda <r$, we have $\left( 2-\frac{\alpha \lambda }{\sqrt{1+\lambda ^2}}\right) |h_{j_1}|+\left( \frac{\alpha }{\sqrt{1+\lambda ^2}}-\alpha \right) |h_{j_0}|>0$. Taking $\lambda \in (0,r)$ in (11) yields that

$$\begin{aligned} f_{S,h}(x)\ge \Vert h\Vert _1-\alpha |h_{j_0}|\ge f_{\overline{S},h}(x^*). \end{aligned}$$

The next proof is similar to the case of $(*)$ due to $|\overline{S}|>1$. $\square$

1.9 Proof of Lemma 8

Proof

Since $|\langle x,y\rangle |\le \Vert x\Vert _2\Vert y\Vert _2=\Vert y\Vert _2$, the infimum of $\langle x,y\rangle$ exists. In addition, $\langle x,y\rangle =\Vert x\Vert _2\Vert y\Vert _2cos \theta =\Vert y\Vert _2cos \theta$, where $\theta$ is the angle between x and y. Hence, the larger $\theta$ is, the smaller $\langle x,y\rangle$ is. Obviously, the closer x is to an axis, the larger $\theta$ is. Thus, the infimum of $\langle x,y\rangle$ is obtained when x is on a certain number of axis. When x takes the unit vector on the number axis, $\Vert y\Vert _2cos \theta$ is the projection on this number axis, and the minimum value is equal to $\min \limits _{i\in R^n}y_i$. Therefore, the conclusion holds. $\square$

1.10 Proof of Proposition 1

Proof

For any $k-$sparse vector x, set $\textrm{supp}(x)=S$, we have $\Vert h_S\Vert _1\le \Vert h_T\Vert _1$, and $\Vert h_{\overline{S}}\Vert _1\ge \Vert h_{\overline{T}}\Vert _1$. So, if $S\subset T$, $\Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\alpha \min \limits _{i\in S}|h_i|\ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\alpha \min \limits _{i\in T}|h_i|$. If $S\not \subset T$

$$\begin{aligned}&\Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\alpha \min \limits _{i\in S}|h_i|\\&\quad = \Vert h_{\overline{S}}\Vert _1-\left( \Vert h_S\Vert _1-\min \limits _{i\in S}|h_i|\right) +(\alpha -1)\min \limits _{i\in S}|h_i|\\&\ge \Vert h_{\overline{T}}\Vert _1-\left( \Vert h_T\Vert _1-\min \limits _{i\in T}|h_i|\right) +(\alpha -1)\min \limits _{i\in S}|h_i|\\&\quad \ge \Vert h_{\overline{T}}\Vert _1-\left( \Vert h_T\Vert _1-\min \limits _{i\in T}|h_i|\right) +(\alpha -1)\min \limits _{i\in T}|h_i|\\&\quad = \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\alpha \min \limits _{i\in T}|h_i|. \end{aligned}$$

Combined with the above two situations, we always have

$$\begin{aligned} \begin{aligned} \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\alpha \min \limits _{i\in S}|h_i| \ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\alpha \min \limits _{i\in T}|h_i|. \end{aligned} \end{aligned}$$

(19)

(13) and (19) yield

$$\begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle =\Vert h_{\overline{S}}\Vert _1+\sum \limits _{i\in S}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) \textrm{sign}(x_i)h_i\\&\ge \Vert h_{\overline{S}}\Vert _1-\sum \limits _{i\in S}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|=\Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\sum \limits _{i\in S}\frac{\alpha |x_i|}{\Vert x\Vert _2}|h_i|\\&\ge \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\sum \limits _{i\in S}\frac{\alpha |x_i|}{\Vert x\Vert _2}\min \limits _{i\in S}|h_i|= \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\frac{\alpha \Vert x\Vert _1}{\Vert x\Vert _2}\min \limits _{i\in S}|h_i|\\&\ge \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\alpha \min \limits _{i\in S}|h_i|\ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\alpha \min \limits _{i\in T}|h_i| >0. \end{aligned}$$

From Theorem 1, we know k-sparse vector x can be recovered from Ax via $\ell _1-\alpha \ell _2$ local minimization. $\square$

1.11 Proof of Proposition 2

Proof

If an arbitrary k-sparse vector x can be recovered from Ax via $\ell _1-\alpha \ell _2$ local minimization, then from Theorem 1, we have, for all k-sparse vector x, (9) holds for all $h\in \textrm{ker} A$ and $\Vert h\Vert _2=1$ with $\textrm{supp}(x)=S$. Combining Lemma 7, we get $f_{T,h}(x^*)>0$ with arbitrary $x^*\ne 0$, $\textrm{supp}(x^*)=T$. Since $\alpha \le 1$, $T_2(x^*)=\emptyset$, $T_1(x^*)=T$, So

$$\begin{aligned} f_{T,h}(x^*)=\Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T}\left( 1-\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|=\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\sum \limits _{i\in T}\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}|h_i|>0 \end{aligned}$$

for all $x^*$ with $\textrm{supp}(x^*)=T$. From Lemma 8,

$$\begin{aligned} \textrm{inf}\left( \sum \limits _{i\in T}\frac{\alpha |x^*_i|}{\Vert x^*\Vert _2}|h_i|\right) =\alpha \, \textrm{inf}\left( \sum \limits _{i\in T}\frac{|x^*_i|}{\Vert x^*\Vert _2}|h_i|\right) =\alpha \min \limits _{i\in T}|h_i|. \end{aligned}$$

Hence, the theorem holds. $\square$

1.12 Proof of Theorem 2

Proof

Necessity is Proposition 2. Now, we prove the sufficiency. Since $\Vert x\Vert _0>1$, we have $\frac{\Vert x\Vert _1}{\Vert x\Vert _2}>1$. If $\min \limits _{i\in S}|h_i|>0$, then from the proof of Proposition 1 and (19), we have

$$\begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle \ge \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\sum \limits _{i\in S}\frac{\alpha |x_i|}{\Vert x\Vert _2}\min \limits _{i\in S}|h_i|\\&\ge \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\frac{\alpha \Vert x\Vert _1}{\Vert x\Vert _2}\min \limits _{i\in S}|h_i|>\Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\alpha \min \limits _{i\in S}|h_i|\\&\ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\alpha \min \limits _{i\in T}|h_i|\ge 0. \end{aligned}$$

If $\min \limits _{i\in S}|h_i|=0$, $\max \limits _{i\in S}|h_i|>0$, then

$$\begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle \ge \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\sum \limits _{i\in S}\frac{\alpha |x_i|}{\Vert x\Vert _2}\min \limits _{i\in S}|h_i|\\&> \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1= \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\alpha \min \limits _{i\in S}|h_i|\\&\ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+\alpha \min \limits _{i\in T}|h_i|\ge 0. \end{aligned}$$

If $\max \limits _{i\in S}|h_i|=0$, then

$$\begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2},\alpha h\right\rangle \ge \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1+\sum \limits _{i\in S}\frac{\alpha |x_i|}{\Vert x\Vert _2}\min \limits _{i\in S}|h_i|\\&= \Vert h_{\overline{S}}\Vert _1-\Vert h_S\Vert _1=\Vert h_{\overline{S}}\Vert _1=\Vert h\Vert _1\ge \Vert h\Vert _2=1>0. \end{aligned}$$

According to Theorem 1, the conclusion of the theorem is true. $\square$

1.13 Proof of Theorem 3

Proof

Necessity is Proposition 2. Now, we prove sufficiency. We prove it in two cases.

$(*)$. $\Vert x\Vert _0>1$. This case is similar to the proof of Theorem 2.

$(**)$. $\Vert x\Vert _0=1$. Set $\textrm{supp}(x)=S=\{j_0\}$. If $h_{j_0}=0$, then

$$\begin{aligned} \sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2}, h\right\rangle =\Vert h_{\overline{S}}\Vert _1=\Vert h\Vert _1\ge \Vert h\Vert _2=1>0. \end{aligned}$$

If $h_{j_0}\ne 0$, then

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2}, h\right\rangle \\&\quad =\textrm{sign}(x_{j_0})h_{j_0}+\Vert h_{\overline{S}}\Vert _1-\textrm{sign}(x_{j_0})h_{j_0}\\&\quad =\Vert h_{\overline{S}}\Vert _1=\Vert h_{[n]\backslash {j_0}}\Vert _1>0. \end{aligned} \end{aligned}$$

The last inequality is because of the assumption $\Vert h\Vert _0>1$, which is normal, because $\Vert h\Vert _0=1$ if and only if A contains a column of 0 vectors, such A is not allowed. Theorem 1 demonstrates that the conclusion of this theorem is valid. $\square$

1.14 Proof of Lemma 9

Proof

This proof is similar to that of Lemma 2. However, it should be noted that there exists an $i\in T$ such that $|h_i|\ne 0$. Hence, at least one of the following two inequalities is strict:

$$\begin{aligned}{} & {} \sum \limits _{i\in T_1(x)}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|\le \sum \limits _{i\in T_1(x^*)}\left( 1-\frac{\alpha |x^*(x)_i|}{\Vert x^*(x)\Vert _2}\right) |h_i|,\\{} & {} \sum \limits _{i\in T_2(x)}\left( \frac{\alpha |x_i|}{\Vert x\Vert _2}-1\right) |h_i|\le \sum \limits _{i\in T_2(x^*)}\left( \frac{\alpha |x^*(x)_i|}{\Vert x^*\Vert _2}-1\right) |h_i|. \end{aligned}$$

Therefore, our lemma conclusion is a strict inequality. $\square$

1.15 Proof of Lemma 10

Proof

Since $T_2(x)=\emptyset$, we know $|T|>1$. Suppose $T=\{i_1,i_2,\cdots i_k\}$ satisfying $|h_{i_1}|\le |h_{i_2}|\le \cdots \le |h_{i_k}|$, $0\le \frac{1}{\alpha }-\frac{|x_{j_1}|}{\Vert x\Vert _2}\le \frac{1}{\alpha }-\frac{|x_{j_2}|}{\Vert x\Vert _2}\le \cdots \le \frac{1}{\alpha }-\frac{|x_{j_k}|}{\Vert x\Vert _2}$, where $\{j_1,j_2,\cdots ,j_k\}$ is a rearrangement of elements in T. We set $\overline{x}_{i_p}=x_{j_p}$ where $p\in [k]$, Otherwise, $\overline{x}_i=0$. From the definition of $\overline{x}$, we have $\textrm{supp}(x)=\textrm{supp}(\overline{x})$, $T_2(\overline{x})=\emptyset$ and the ordering inequality shows $g_{T,h}(\overline{x})\ge g_{T,h}(x)$. Now, let us keep $\overline{x_{i_1}}$ constant and let $|\overline{x}_i|$ for $i\in T\backslash \{i_1\}$ decrease but not to 0. We assume that after this change, $\overline{x}$ becomes $x^*$. Obviously, for all $i\in T\backslash \{i_1\}$, when $|x^*_i|\rightarrow 0$, we have $\frac{|x^*_i|}{\Vert x^*\Vert _2}\rightarrow 0$, $0\le \frac{1}{\alpha }-\frac{|\overline{x}_i|}{\Vert \overline{x}\Vert _2}\le \frac{1}{\alpha }-\frac{|x^*_i|}{\Vert x^*\Vert _2}$ and $\frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}\rightarrow 1>\frac{1}{\alpha }$. This shows that, when $x_i$ is sufficiently small for all $i\in T\backslash \{i_1\}$, we have $T_2(x^*)=\{i_1\}$, $T_1(x^*)=T\backslash \{i_1\}$ and $\frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{|\overline{x}_i|}{\Vert \overline{x}\Vert _2}\le 0$. Hence,

$$\begin{aligned} g_{T,h}(x)-g_{T,h}(x^*){} & {} \le g_{T,h}(\overline{x})-g_{T,h}(x^*)\nonumber \\= & {} \sum \limits _{i\in T_1(\overline{x})}\left( \frac{1}{\alpha }-\frac{|\overline{x}_i|}{\Vert \overline{x}\Vert _2}\right) |h_i|-\sum \limits _{i\in T_1(x^*)}\left( \frac{1}{\alpha }-\frac{|x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|\nonumber \\{} & {} -\sum \limits _{i\in T_2(x^*)}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_i|\nonumber \\= & {} \sum \limits _{i\in T\backslash \{i_1\}}\left( \frac{1}{\alpha }-\frac{|\overline{x}_i|}{\Vert \overline{x}\Vert _2}\right) |h_i|+\left( \frac{1}{\alpha }-\frac{|\overline{x}_{i_1}|}{\Vert \overline{x}\Vert _2}\right) |h_{i_1}|\nonumber \\{} & {} -\sum \limits _{i\in T\backslash \{i_1\}}\left( \frac{1}{\alpha }-\frac{|x^*_i|}{\Vert x^*\Vert _2}\right) |h_i|-\left( \frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}-\frac{1}{\alpha }\right) |h_{i_1}|\nonumber \\= & {} \sum \limits _{i\in T\backslash \{i_1\}}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{|\overline{x}_i|}{\Vert \overline{x}\Vert _2}\right) |h_i|+\left( \frac{2}{\alpha }-\frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}-\frac{|\overline{x}_{i_1}|}{\Vert \overline{x}\Vert _2}\right) |h_{i_1}|\nonumber \\{} & {} \le \sum \limits _{i\in T\backslash \{i_1\}}\left( \frac{|x^*_i|}{\Vert x^*\Vert _2}-\frac{|\overline{x}_i|}{\Vert \overline{x}\Vert _2}\right) |h_{i_1}|+\left( \frac{2}{\alpha }-\frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}-\frac{|\overline{x}_{i_1}|}{\Vert \overline{x}\Vert _2}\right) |h_{i_1}|\nonumber \\= & {} \left( \frac{2}{\alpha }-\frac{\Vert \overline{x}\Vert _1}{\Vert \overline{x}\Vert _2}-\frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}+\sum \limits _{i\in T\backslash \{i_1\}}\frac{|x_i^*|}{\Vert x^*\Vert _2}\right) |h_{i_1}|.\nonumber \\ \end{aligned}$$

(20)

In addition, since $\frac{\Vert \overline{x}\Vert _1}{\Vert \overline{x}\Vert _2}>1$, $\frac{2}{\alpha }\in [1,2)$, we have

$$\begin{aligned} \lim \limits _{x_i^*\rightarrow 0,\forall i\in T\backslash \{i_1\}}\frac{2}{\alpha }-\frac{\Vert \overline{x}\Vert _1}{\Vert \overline{x}\Vert _2}-\frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}+\sum \limits _{i\in T\backslash \{i_1\}}\frac{|x_i^*|}{\Vert x^*\Vert _2}=\frac{2}{\alpha }-\frac{\Vert \overline{x}\Vert _1}{\Vert \overline{x}\Vert _2}-1+0<0. \end{aligned}$$

By the limit sign preserving property, there exists an $x^*$ such that $\frac{2}{\alpha }-\frac{\Vert \overline{x}\Vert _1}{\Vert \overline{x}\Vert _2}-\frac{|x^*_{i_1}|}{\Vert x^*\Vert _2}+\sum \limits _{i\in T\backslash \{i_1\}}\frac{|x_i^*|}{\Vert x^*\Vert _2}<0$. Combined with (20), $g_{T,h}(x^*)\ge g_{T,h}(x)$, that is $f_{T,h}(x^*)\le f_{T,h}(x)$. $\square$

1.16 Proof of Proposition 3

Proof

For an arbitrary k-sparse vector $x\in R^n$, set $S=\textrm{supp}(x)$. For all $h\in \textrm{ker} A$ and $\Vert h\Vert _2=1$, $T=\textrm{supp}(h_{\textrm{max}(k)})$, from Lemma 5, Lemma 6 and the proof process of case $(**)$ in Lemma 7, there exists an $x^*$, $\textrm{supp}(x^*)=T$, $|T_2(x^*)|=1$ such that $f_{S,h}(x)\ge f_{T,h}(x^*)$. Hence,

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2}, \alpha h\right\rangle \\&=\Vert h_{\overline{S}}\Vert _1+\sum \limits _{i\in S}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) \textrm{sign}(x_i)h_i\\&\ge f_{S,h}(x)\ge f_{T,h}(x^*). \end{aligned} \end{aligned}$$

(21)

In the case of $k=1$, $T_2(x^*)=T$ and $|T_2(x^*)|=1$. Set $T_2(x^*)=\{j_0\}$ then

$$\begin{aligned} \begin{aligned} f_{T,h}(x^*)&=\Vert h_{\overline{T}}\Vert _1-\left( \frac{\alpha |x_{j_0}|}{\Vert x\Vert _2}-1\right) |h_{j_0}|\\&=\Vert h_{\overline{T}}\Vert _1-(\alpha -1)|h_{j_0}|\\&=\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )|h_{j_0}|. \end{aligned} \end{aligned}$$

(22)

where we use the fact $\Vert h_T\Vert _1=|h_{j_0}|$. Combining (15), (21) and (22), we get

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2}, \alpha h\right\rangle \\&\ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )|h_{j_0}|\\&=\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )\min \limits _{i\in T}|h_i|>0. \end{aligned} \end{aligned}$$

(23)

From Theorem 1, the conclusion is true in the case of $k=1$.

In the case of $k>1$, we define a set $\Omega (x^*)=\{x|\textrm{supp}(x)=\textrm{supp}(x^*),T_2(x)=T_2(x^*)\}$. Setting $T_2(x^*)=\{j_0\}$, by (15) and (21) we yield

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2}, \alpha h\right\rangle \ge f_{T,h}(x^*)\ge \inf \limits _{x\in \varOmega (x^*)}\,f_{T,h}(x)\\&=\inf \limits _{x\in \varOmega (x^*)}\left\{ \Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T\backslash \{j_0\}}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|-\left( \frac{\alpha |x_{j_0}|}{\Vert x\Vert _2}-1\right) |h_{j_0}|\right\} \\&=\Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T\backslash \{j_0\}}|h_i|-(\alpha -1)|h_{j_0}|\\&=\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )|h_{j_0}|\\&\ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )\min \limits _{i\in T}|h_i|>0. \end{aligned} \end{aligned}$$

(24)

Theorem 1 shows that this conclusion holds for $k>1$. Therefore, we finish the proof. $\square$

1.17 Proof of Proposition 4

Proof

From Theorem 1 and Lemma 7, if an arbitrary k-sparse vector $x\in R^n$ can be recovered from Ax via $\ell _1-\alpha \ell _2$ local minimization, then $f_{T,h}(x^*)>0$ holds for all $h\in \textrm{ker} A$, $\Vert h\Vert _2=1$ with $T=\textrm{supp}(h_{\textrm{max}(k)})$ and arbitrary $x^*\ne 0$ satisfying $\textrm{supp}(x^*)=T$. Setting $j_0=\textrm{argmin}_{i\in T}|h_i|$, we define a vector x(h) as

$$\begin{aligned} x(h)_i= {\left\{ \begin{array}{ll} 1&{} i=j_0\\ \varepsilon &{} i\in T\backslash \{j_0\}\\ 0&{} i\in \overline{T}\\ \end{array}\right. }, \end{aligned}$$

where $\varepsilon$ is sufficiently small so that $S_1(x(h))= T\backslash \{j_0\}$ and $S_2(x(h))= \{j_0\}$. We define a set $\Omega (x(h))=\{x|\textrm{supp}(x)=\textrm{supp}(x(h)),T_2(x)=T_2(x(h))\}$, then

$$\begin{aligned} \begin{aligned}&\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )\min \limits _{i\in T}|h_i|\\&=\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )|h_{j_0}|\\&=\Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T\backslash \{j_0\}}|h_i|-(\alpha -1)|h_{j_0}|\\&=\inf \limits _{x\in \varOmega (x(h))}\left\{ \Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T\backslash \{j_0\}}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|-\left( \frac{\alpha |x_{j_0}|}{\Vert x\Vert _2}-1\right) |h_{j_0}|\right\} \\&=\inf \limits _{x\in \varOmega (x(h))}\,f_{T,h}(x)\ge 0, \end{aligned} \end{aligned}$$

The last inequality is because $f_{T,h}(x)>0$ holds for all $x\ne 0$ with $\textrm{supp}(x)=T$. $\square$

1.18 Proof of Theorem 4

Proof

Necessity is Proposition 4. Now, we prove sufficiency. First, similar to Proposition 3, for an arbitrary k-sparse vector $x\in R^n$ with $S=\textrm{supp}(x)$, there exists an $x^*$, $\textrm{supp}(x^*)=T$, $|T_2(x^*)|=1$ such that $f_{S,h}(x)\ge f_{T,h}(x^*)$. Set $T_2(x^*)=\{j_0\}$, and, we define a vector $x^{**}$ as

$$\begin{aligned} x^{**}_i= {\left\{ \begin{array}{ll} 0.5x^*_i&{} i\ne j_0\\ x^*_i&{} i=j_0\\ \end{array}\right. }, \end{aligned}$$

We know that $T_1(x^*)=T_1(x^{**})$, $T_2(x^*)=T_2(x^{**})$. Since $\Vert x\Vert _0>1$, $|T_2(x^{**})|=1$, we know $|T_1(x^{**})|\ne \emptyset$. From Lemma 9, we have $f_{T,h}(x^{**})<f_{T,h}(x^*)$. Next, we define a set $\Omega (x^{**})=\{x|\textrm{supp}(x)=\textrm{supp}(x^{**}),T_2(x)=T_2(x^{**})\}$, then

$$\begin{aligned} \begin{aligned}&\sum \limits _{i\in S}\textrm{sign}(x_i)h_i+\Vert h_{\overline{S}}\Vert _1-\left\langle \frac{x}{\Vert x\Vert _2}, \alpha h\right\rangle \ge f_{T,h}(x^*)>f_{T,h}(x^{**})\ge \inf \limits _{x\in \varOmega (x^{**})}\,f_{T,h}(x)\\&\quad =\inf \limits _{x\in \varOmega (x^{**})}\left\{ \Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T\backslash \{j_0\}}\left( 1-\frac{\alpha |x_i|}{\Vert x\Vert _2}\right) |h_i|-\left( \frac{\alpha |x_{j_0}|}{\Vert x\Vert _2}-1\right) |h_{j_0}|\right\} \\&\quad =\Vert h_{\overline{T}}\Vert _1-\sum \limits _{i\in T\backslash \{j_0\}}|h_i|-(\alpha -1)|h_{j_0}|\\&\quad =\Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )|h_{j_0}|\\&\quad \ge \Vert h_{\overline{T}}\Vert _1-\Vert h_T\Vert _1+(2-\alpha )\min \limits _{i\in T}|h_i|\ge 0. \end{aligned} \end{aligned}$$

Because of Theorem 1, the result of the theorem is true. $\square$