Matching pursuit with unbounded parameter domains

Abstract

In various applications, the adoption of optimal energy matching pursuit with dictionary elements is common. When the dictionary elements are indexed by parameters within a bounded region, exhaustion-type algorithms can be employed. This article aims to investigate a process that converts the optimal parameter selection in unbounded regions to a bounded and closed (compact) sub-domain. Such a process provides accessibility for energy matching pursuit in a wide range of applications. The paper initially focuses on the open unit disc and the upper-half complex plane, introducing adaptive Fourier decomposition as the underlying methodology. It then extends this concept to general Hilbert spaces with a dictionary and Bochner spaces for random signals. Computational examples are included to illustrate the concepts presented.

Appendix. Proof of Lemma 3

The proof is based on standard techniques of harmonic analysis. Since $f\in H^2(\textbf{C}^+),$ the corresponding Paley-Wiener Theorem asserts that $\textrm{supp}\hat{f}\subset [0,\infty ).$ Due to absolute continuity of integration, there exist $0<\delta _1< N_1 <\infty ,$ such that

$$\begin{aligned} \Vert \chi _{[\delta _1,N_1]}\hat{f}-\hat{f}\Vert <\varepsilon /2,\end{aligned}$$

(1)

where for any set $A\subset \textbf{R},$ $\chi _A$ denotes the indicator function of A. Let $\phi$ be a $C_0^\infty$ function on the line with support in $[-\delta _1/2,\delta _1/2]$ such that

$$\begin{aligned} \Vert \phi *(\chi _{[\delta _1,N_1]}\hat{f})-\chi _{[\delta _1,N_1]}\hat{f}\Vert <\varepsilon /16.\end{aligned}$$

(2)

Such function $\phi$ may be constructed by using the approximation to identity process via dilation and translation of any function in $C_0^\infty$ with integral 1 (See, for instance, [31]). For the algorithm reason, we enclose the details concerning the parameter estimations.

Let $\psi$ be a non-negative $L^1$-function of integral 1 with compact support in $[-1/2,1/2].$ We show that there exists ${0<t_0\le \delta _1}$ such that for all t satisfying $0<t\le t_0$, there holds

$$\begin{aligned} \Vert \psi _t*F-F\Vert <\varepsilon /16,\end{aligned}$$

(3)

where we have used $F=(\chi _{[\delta _1,N_1]}\hat{f})$ for short, and $\psi _t$ the norm-1 preserving dilation of $\psi : \psi _t(x)=\frac{1}{t}\psi (\frac{x}{t}).$ Once such $t_0$ is found, we take $\phi =\psi _{t_0}$ in (31) and let $g=(\phi *(\chi _{[\delta _1,N_1]}\hat{f}))^\vee .$ Owing to the Plancherel Theorem,

$$\begin{aligned} \Vert \psi _t*F-F\Vert ^2 =\Vert ((\psi _t\hat{)}-1)\hat{F}\Vert ^2 =\left( \int _{-N_2}^{N_2} +\int _{-\infty }^{-N_2}+\int _{N_2}^\infty \right) |G_t(\xi )|^2d\xi , \end{aligned}$$

where $G_t=((\psi _t\hat{)}-1)\hat{F}.$ Since $(\psi _t\hat{)}$ is bounded by 1,  we can first determine $N_2$ large enough such that

$$\begin{aligned} \int _{N_2}^\infty |G_t(\xi )|^2d\xi \le 4\int _{N_2}^\infty |\hat{F}(\xi )|^2d\xi < \frac{\varepsilon }{64}.\end{aligned}$$

(4)

The integral from $-\infty$ to $-N_2$ is similarly estimated, involving maybe an adjustment of $N_2.$

Note that $(\psi _t\hat{)}(\xi )=\hat{\psi }(t\xi ),\ \hat{\psi }(0)=1,$ and $\hat{\psi } (\xi )$ is continuous at $\xi =0.$ When $N_2$ is fixed, $t_0$ can be chosen small enough to satisfy $t_0\le \delta _1,$ and for $0<t\le t_0,$

$$\begin{aligned} |\hat{\psi }(t\xi )-1|< \frac{\varepsilon }{128\Vert F\Vert _2^2}\end{aligned}$$

(5)

uniformly for $\xi \in [-N_2,N_2].$ Then we have

$$\begin{aligned} \int _{-N_2}^{N_2}|G_t(\xi )|^2d\xi< \frac{\varepsilon }{128\Vert F\Vert _2^2}\int _{-N_2}^{N_2}|\hat{F}(\xi )|^2d\xi < \frac{\varepsilon }{128}.\end{aligned}$$

(6)

In summary, (32) holds.

Taking $\phi =\psi _{t_0}$ and $g=(\phi *(\chi _{[\delta _1,N_1]}\hat{f}))^\vee .$ The Titchmarsh Convolution Theorem asserts that $\phi *(\chi _{[\delta _1,N_1]}\hat{f}))$ has support contained in $[\delta _1/2,N_1+\delta _1/2].$ Using the Paley-Wiener theorem, again, we assert that $g=(\phi *(\chi _{[\delta _1,N_1]}\hat{f}))^\vee \in H^2(\textbf{C}^+).$ Using the Riemann-Lebesgue lemma to the $L^1(\textbf{R})$ function $\phi *(\chi _{[\delta _1,N_1]}\hat{f})$, we have that $\hat{g}$ is bounded and tending to zero at $\infty .$ It also has smoothness to any degree. From the construction, it is easy to see that $\Vert g-f\Vert _{H^2(\textbf{C}^+)}<\varepsilon .$ We next show that g has the properties claimed in (15). We take the case $j=0$ in (15) as the rest cases are similar. Note that for $z\in \textbf{C}^+, t\in \textbf{R}^+, |\textrm{e}^{\textrm{i}tz}|\le 1.$ For $N=N_1+\delta _1/2,$ owing to the boundedness of $\hat{g}$, there exists $L_1>0,$

$$\begin{aligned} |g(z)|= & {} \frac{1}{2\pi }\left| \int _0^N\textrm{e}^{\textrm{i}tz}\hat{g}(t)dt \right| < L_1. \end{aligned}$$

(7)

By using integration by parts, owing to the smoothness of $\hat{g},$ there exists $L_2>0,$

$$\begin{aligned} |g(z)|= & {} \frac{1}{2\pi }\left| \int _0^N\textrm{e}^{\textrm{i}tz}\hat{g}(t)dt \right| \nonumber \\= & {} \frac{1}{2\pi }\left| \int _0^N\hat{g}(t)d\left( \frac{\textrm{e}^{\textrm{i}tz}}{\textrm{i}z}\right) \right| \nonumber \\< & {} \frac{L_2}{|z|}, \qquad z\in \textbf{C}^+. \end{aligned}$$

(8)

Therefore, for $M_0=L_1+L_2,$

$$\begin{aligned} |g(z)|< \frac{M_0}{1+|z|}, \qquad z\in \textbf{C}^+. \end{aligned}$$

The proof of the lemma is complete.

A more detailed analysis shows that the $M_j$’s are dependent of $\epsilon$, and, when $\epsilon$ becomes smaller, the $M_j$’s usually become larger.