Jackknife model averaging for linear regression models with missing responses

Abstract

We consider model averaging estimation problem in the linear regression model with missing response data, that allows for model misspecification. Based on the ‘complete’ data set for the response variable after inverse propensity score weighted imputation, we construct a leave-one-out cross-validation criterion for allocating model weights, where the propensity score model is estimated by the covariate balancing propensity score method. We derive some theoretical results to justify the proposed strategy. Firstly, when all candidate outcome regression models are misspecified, our procedures are proved to achieve optimality in terms of asymptotically minimizing the squared loss. Secondly, when the true outcome regression model is among the set of candidate models, the resulting model averaging estimators of the regression parameters are shown to be root-n consistent. Simulation studies provide evidence of the superiority of our methods over other existing model averaging methods, even when the propensity score model is misspecified. As an illustration, the approach is further applied to study the CD4 data.

Appendix: Proofs of the main results

For the sake of convenience, we denote the general constant by C whose value could be different at each appearance. Write $\pi _{i}=\pi (X_{i}, \alpha )$ and $\hat{\pi }_{i}=\pi (X_{i}, \hat{\alpha })$ for $i=1,\ldots , n$. Let $A^{(k)}=I_{n}-H^{(k)}$, $T^{(k)}=D^{(k)}-I_{n}$, and $G^{(k)}=T^{(k)}A^{(k)}$. Denote $A(w)=\sum _{k=1}^{K}w_{k}A^{(k)}$, $G(w)=\sum _{k=1}^{K}w_{k}G^{(k)}$, and $M(w)=A'(w)G(w)+G'(w)A(w)+G'(w)G(w)$.

Lemma 1

Suppose that $X_{i}, i=1\ldots ,n$ are a set of i.i.d. random vectors. If condition (C1) is satisfied, then $\hat{\alpha } {\mathop {\longrightarrow }\limits ^{p}} \alpha _{0}$, and $\hat{\alpha } - \alpha _{0}=O_{p}(n^{-1/2})$.

Proof

The proof of $\hat{\alpha } {\mathop {\longrightarrow }\limits ^{p}} \alpha _{0}$ is the same as the proof of Theorem 1 in Guo et al. (2017), $\hat{\alpha } - \alpha _{0}=O_{p}(n^{-1/2})$ is implied by Theorem 3.2 of Newey and McFadden (1994).

Lemma 2

Under condition (C1), we have

$$\begin{aligned} \Vert \hat{Y}-\check{Y} \Vert ^{2}=O_{p}(1). \end{aligned}$$

Proof

Simple calculations yield $\check{y}_{i}-\mu _{0i}=s_{i}$, $\hat{y}_{i}-\mu _{0i}=s_{i}+e_{i}$, where $s_{i}=\dfrac{\delta _{i}}{\pi _{i}}(y_{i}-X_{i}'\beta )+(X_{i}'\beta -\mu _{0i})$, $e_{i}=(1-\dfrac{\delta _{i}}{\hat{\pi }_{i}})X_{i}'(\hat{\beta }_{c}-\beta )+\delta _{i}(\dfrac{1}{\hat{\pi }_{i}}-\dfrac{1}{\pi _{i}})(y_{i}-X_{i}'\beta )$. Therefore, $\hat{y}_{i}-\check{y}_{i}=e_{i}$ and $\Vert \hat{Y}-\check{Y} \Vert ^{2}=\sum _{i=1}^{n}e_{i}^{2}$. By the Taylor series expansion, condition (C1) and Lemma 1, we have

$$\begin{aligned} \max \limits _{1\le i \le n} \left| \frac{1}{\hat{\pi }_{i}}-\frac{1}{\pi _{i}} \right|&=\max \limits _{1\le i \le n} \left| \left\{ \frac{1}{\pi ^{2}(X_{i}, \alpha )}\frac{\partial \pi (X_{i}, \alpha )}{\partial \alpha '}\right\} \Big |_{\alpha =\tilde{\alpha }}\cdot (\hat{\alpha }-\alpha _{0}) \right| \nonumber \\&\le \max \limits _{1\le i \le n}\left\| \frac{1}{\pi ^{2}(X_{i}, \alpha )}\frac{\partial \pi (X_{i}, \alpha )}{\partial \alpha } \Big |_{\alpha =\tilde{\alpha }}\right\| \cdot \Vert \hat{\alpha }-\alpha _{0} \Vert \nonumber \\&=O_{p}\left( n^{-1/2}\right) , \end{aligned}$$

(A.1)

where $\tilde{\alpha }$ is a vector between $\hat{\alpha }$ and $\alpha _{0}$. This together with the facts that $\pi _{i}$ is bounded away from 0 and $\dfrac{\delta _{i}}{\hat{\pi }_{i}} =\dfrac{\delta _{i}}{\pi _{i}}+(\dfrac{\delta _{i}}{\hat{\pi }_{i}} -\dfrac{\delta _{i}}{\pi _{i}})$, it is easy to verify that

$$\begin{aligned} \max \limits _{1\le i \le n} \left| \frac{\delta _{i}}{\hat{\pi }_{i}}\right| \le \max \limits _{1\le i \le n} \left| \frac{1}{\pi _{i}}\right| + \max \limits _{1\le i \le n} \left| \frac{1}{\hat{\pi }_{i}}-\frac{1}{\pi _{i}} \right| =O_{p}(1). \end{aligned}$$

(A.2)

Combining (A.1), (A.2), the fact that $\hat{\beta }_{C}-\beta =O_{p}(n^{-1/2})$ and condition (C1), the lemma is then proved. $\square$

Lemma 3

Suppose that condition (C2) holds, we have

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{A(w)\}\le 1, \end{aligned}$$

(A.3)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{G(w)\}=O_{p}(\bar{p}n^{-1}),\end{aligned}$$

(A.4)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}=O_{p}(\bar{p}n^{-1}). \end{aligned}$$

(A.5)

Proof

$\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{A(w)\}\le \sup \limits _{w\in \mathcal {W}} \displaystyle \sum _{k=1}^{K}w_{k}\bar{\lambda }(A^{(k)})\le \max _{1\le k \le K}\bar{\lambda }(A^{(k)})\le 1.$

Note that $\bar{\lambda }(T^{(k)})=\max \limits _{1\le j \le n}\left\{ (1-h_{jj}^{(k)})^{-1}-1\right\} =\max \limits _{1\le j \le n} \dfrac{h_{jj}^{(k)}}{1-h_{jj}^{(k)}}$. By (A.3) and condition (C2), we have

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{G(w)\}&\le \sup \limits _{w\in \mathcal {W}} \displaystyle \sum _{k=1}^{K}w_{k}\bar{\lambda }(T^{(k)}A^{(k)})\le \max _{1\le k \le K}\bar{\lambda }(T^{(k)})\bar{\lambda }(A^{(k)})\\&\le \max _{1\le k \le K}\max _{1\le j \le n}\frac{h_{jj}^{(k)}}{1-h_{jj}^{(k)}}=O_{p}(\bar{p}n^{-1}). \end{aligned}$$

Combining (A.3) and (A.4), we obtain

$$\begin{aligned} \begin{aligned} \sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}&\le 2\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{A'(w)G(w)\}+\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{G'(w)G(w)\} \\&\le 2\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{A(w)\}\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{G(w)\}+\left[ \sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{G(w)\}\right] ^{2}=O_{p}(\bar{p}n^{-1}). \end{aligned} \end{aligned}$$

$\square$

Proof of Theorem 1

Denote $e=(e_{1}, \ldots , e_{n})'$, $S=(s_{1}, \ldots , s_{n})'$. $\Sigma _{S}=E(SS')$ is a diagonal matrix whose ith diagonal component is

$$\begin{aligned} \sigma _{S,ii}^{2}=E(s_{i}^{2})=\sigma _{i}^{2}/ \pi _{i}+(1/\pi _{i}+1)(X_{i}'\beta -\mu _{0i})^{2}. \end{aligned}$$

(A.6)

Observe that CV(w) can be written as

$$\begin{aligned} CV(w)&= \hat{Y}A'(w)A(w)\hat{Y}+\hat{Y}'M(w)\hat{Y}\\&= \hat{L}_{n}(w)+2\mu _{0}'A'(w)S-2S'H'(w)S+4e'A'(w)S +2\mu _{0}'A'(w)e \\&\quad +2e'A'(w)e+\mu _{0}'M(w)\mu _{0}+2\mu _{0}'M(w)S +2\mu _{0}'M(w)e\\&\quad +S'M(w)S+2S'M(w)e+e'M(w)e+S'S-2S'e-e'e, \end{aligned}$$

where the last three terms are independent of w. Therefore, the claim $\hat{L}_{n}(\hat{w})/\{\inf _{w\in \mathcal {W}}\hat{L}_{n} (w)\}{\mathop {\longrightarrow }\limits ^{p}}1$ is valid if the following hold when $n\rightarrow \infty$:

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|\mu _{0}'A'(w)S|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.7)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|S'H'(w)S|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.8)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|e'A'(w)S|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.9)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|\mu _{0}'A'(w)e|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.10)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|e'A'(w)e|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.11)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|\mu _{0}'M(w)\mu _{0}|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.12)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|\mu _{0}'M(w)S|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.13)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|\mu _{0}'M(w)e|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.14)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|S'M(w)S|/ \check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.15)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|S'M(w)e|/ \check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.16)

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}|e'M(w)e|/\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.17)

and

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}|\hat{L}_{n}(w)/ \check{R}_{n}(w)-1|=o_{p}(1). \end{aligned}$$

(A.18)

According to the definition of $\check{R}_{n}(w)$, it is straightforward to show that

$$\begin{aligned} \check{R}_{n}(w)=\Vert A(w)\mu _{0}\Vert ^{2}+\text{tr}\{H'(w)H(w)\Sigma _{S}\}. \end{aligned}$$

(A.19)

From the Chebyshev’s inequality, Theorem 2 of Whittle (1960), (A.19), conditions (C4) and (C5), for any $\tau >0$, we observe that

$$\begin{aligned} P&\left\{ \sup \limits _{w\in \mathcal {W}}\vert \mu _{0}'A'(w)S\vert /\check{R}_{n}(w)>\tau \right\} \le \sum _{k=1}^{K}P\left\{ \vert \mu _{0}'A'(w_{k}^{0})S\vert > \tau \check{\xi }_{n}\right\} \\&\le \tau ^{-2N}\check{\xi }_{n}^{-2N}\displaystyle \sum _{k=1}^{K}E \{\mu _{0}'A'(w_{k}^{0})S\}^{2N} \le C C_{S}^{N}\tau ^{-2N} \check{\xi }_{n}^{-2N}\displaystyle \sum _{k=1}^{K} \Vert A(w_{k}^{0})\mu _{0}\Vert ^{2N} \\&\le C C_{S}^{N}\tau ^{-2N}\check{\xi }_{n}^{-2N} \displaystyle \sum _{k=1}^{K}\left\{ \check{R}_{n} (w_{k}^{0})\right\} ^{N} =o_{p}(1). \end{aligned}$$

Thus, (A.7) is proved. To show (A.8), it suffices to prove that

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\vert S'H'(w)S-\text {tr}\{H'(w)\Sigma _{S}\}\vert /\check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.20)

and

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\vert \text {tr}\{H'(w) \Sigma _{S}\}\vert /\check{R}_{n}(w)=o_{p}(1). \end{aligned}$$

(A.21)

Similar to the proof of (A.7), we see, for any $\tau >0$, that

$$\begin{aligned} P&\left\{ \sup \limits _{w\in \mathcal {W}}\vert S'H'(w)S-\text {tr}\{H'(w)\Sigma _{S}\}\vert / \check{R}_{n}(w)>\tau \right\} \\&\le \displaystyle \sum _{k=1}^{K}P\left\{ \vert S'H'(w_{k}^{0})S -\text {tr}\{H'(w_{k}^{0})\Sigma _{S}\}\vert >\tau \check{\xi }_{n}\right\} \\&\le \tau ^{-2N}\check{\xi }_{n}^{-2N}\displaystyle \sum _{k=1}^{K}E \left[ S'H'(w_{k}^{0})S-\text {tr}\left\{ H'(w_{k}^{0}) \Sigma _{S}\right\} \right] ^{2N} \\&\le CC_{S}^{N}\tau ^{-2N}\check{\xi }_{n}^{-2N} \displaystyle \sum _{k=1}^{K}\left[ \text {tr} \left\{ H'(w_{k}^{0})\Sigma _{S}H(w_{k}^{0})\right\} \right] ^{N} \\&\le CC_{S}^{N}\tau ^{-2N}\check{\xi }_{n}^{-2N} \displaystyle \sum _{k=1}^{K}\left[ \check{R}_{n} (w_{k}^{0})\right] ^{N}=o_{p}(1), \end{aligned}$$

(A.20) is thus obtained. On the other hand, it follows from

$$\begin{aligned} \vert \text {tr}(H^{(k)}{'}\Sigma _{S})\vert&=\frac{1}{2}\vert \text {tr}(H^{(k)}{'}\Sigma _{S}+\Sigma _{S}H^{(k)})\vert \\&\le \frac{1}{2}\bar{\lambda }(H^{(k)}{'}\Sigma _{S} +\Sigma _{S}H^{(k)})\text {rank}(H^{(k)}{'}\Sigma _{S} +\Sigma _{S}H^{(k)}) \\&\le 2\bar{\lambda }(H^{(k)}) \bar{\lambda }(\Sigma _{S}) \text {rank}(H^{(k)}{'}\Sigma _{S}) \le 2C_{S}\cdot \bar{p} \end{aligned}$$

and condition (C6) that

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\vert \text {tr} (H'(w)\Sigma _{S})\vert /\check{R}_{n}(w)\le \check{\xi }_{n}^{-1}\max _{1\le k \le K}\left| \text {tr}(H^{(k)}{'}\Sigma _{S})\right| \le 2C_{S}\cdot \bar{p}\check{\xi }_{n}^{-1}=o_{p}(1). \end{aligned}$$

So (A.21) holds. We have proved (A.8). By condition (C4), $E\{(S'S)^{1/2}\}\le \{E(S'S)\}^{1/2}\le \sqrt{nC_{S}}$, and hence $(S'S)^{1/2}=O_{p}(\sqrt{n})$ and $S'S=O_{p}(n)$. This together with Lemma 2, Lemma 3 and conditions (C3) and (C6) yield

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}|e'A'(w)S|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1}\sup \limits _{w\in \mathcal {W}}\{S'A(w)ee'A'(w)S\}^{1/2}\\&\le \check{\xi }_{n}^{-1} (e'e)^{1/2}(S'S)^{1/2}\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{A(w)\}\\&\le \check{\xi }_{n}^{-1}O_{p}(1)O_{p}(\sqrt{n})\cdot 1=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|\mu _{0}'M(w)S|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (\mu _{0}'\mu _{0})^{1/2}(S'S)^{1/2}\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}\\&=\check{\xi }_{n}^{-1}O(\sqrt{n})O_{p} (\sqrt{n})O_{p}(\bar{p}n^{-1})=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|S'M(w)S|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (S'S)\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}\\&=\check{\xi }_{n}^{-1}O_{p}(n)O_{p}(\bar{p}n^{-1})=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|S'M(w)e|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (S'S)^{1/2}(e'e)^{1/2}\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}\\&\le \check{\xi }_{n}^{-1}O_{p}(\sqrt{n}) O_{p}(1)O_{p}(\bar{p}n^{-1})=o_{p}(1). \end{aligned}$$

So (A.9), (A.13), (A.15) and (A.16) are correct. Similar to the above proof steps, it can be demonstrated that

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}|\mu _{0}'A'(w)e|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1}(\mu _{0}'\mu _{0})^{1/2}(e'e)^{1/2} \sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{A(w)\}\\&\le \check{\xi }_{n}^{-1} O_{p}(\sqrt{n})O_{p}(1)\cdot 1=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|e'A'(w)e|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (e'e)\sup \limits _{w\in \mathcal {W}} \bar{\lambda }\{A(w)\}\le \check{\xi }_{n}^{-1} O_{p}(1)\cdot 1=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|\mu _{0}'M(w)\mu _{0}|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (\mu _{0}'\mu _{0}) \sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}\\&=\check{\xi }_{n}^{-1}O(n)O_{p}(\bar{p}n^{-1})=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|\mu _{0}'M(w)e|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (\mu _{0}'\mu _{0})^{1/2} (e'e)^{1/2}\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}\\&=\check{\xi }_{n}^{-1}O(\sqrt{n})O_{p}(1)O_{p}(\bar{p}n^{-1})=o_{p}(1),\\ \sup \limits _{w\in \mathcal {W}}|e'M(w)e|/\check{R}_{n}(w)&\le \check{\xi }_{n}^{-1} (e'e)\sup \limits _{w\in \mathcal {W}}\bar{\lambda }\{M(w)\}\\&\le \check{\xi }_{n}^{-1}O_{p}(1)O_{p}(\bar{p}n^{-1})=o_{p}(1). \end{aligned}$$

Therefore, (A.10), (A.11), (A.12), (A.14) and (A.17) are valid.

Next we prove (A.18). Because

$$\begin{aligned} \hat{L}_{n}(w)&=\Vert \mu _{0}-\hat{\mu }(w)\Vert ^{2} =\Vert \mu _{0}-\check{\mu }(w)+\check{\mu }(w)-\hat{\mu }(w)\Vert ^{2}\\&\le \check{L}_{n}(w)+2\{\check{L}_{n}(w)\}^{1/2} \Vert \check{\mu }(w)-\hat{\mu }(w)\Vert +\Vert \check{\mu }(w) -\hat{\mu }(w)\Vert ^{2}, \end{aligned}$$

and

$$\begin{aligned} \Vert \check{\mu }(w)-\hat{\mu }(w)\Vert ^{2}=\Vert H(w)\check{Y} -H(w)\hat{Y}\Vert ^{2}\le \Vert \check{Y}-\hat{Y}\Vert ^{2}, \end{aligned}$$

it is sufficient to verify that

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}|\check{L}_{n}(w)/ \check{R}_{n}(w)-1|=o_{p}(1), \end{aligned}$$

(A.22)

and

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}} \Vert \check{Y}-\hat{Y}\Vert ^{2}/\check{R}_{n}(w)=o_{p}(1). \end{aligned}$$

(A.23)

Additionally, recognize that

$$\begin{aligned} |\check{L}_{n}(w)-\check{R}_{n}(w)|=\left| \Vert H(w)S\Vert ^{2}-\text{tr}\{H'(w)H(w)\Sigma _{S}\} -2\mu _{0}'A'(w)H(w)S\right| , \end{aligned}$$

so (A.22) is implied by

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\left| \Vert H(w)S\Vert ^{2}-\text{tr}\{H'(w)H(w)\Sigma _{S}\}\right| / \check{R}_{n}(w)=o_{p}(1), \end{aligned}$$

(A.24)

and

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\left| \mu _{0}'A'(w)H(w) S\right| /\check{R}_{n}(w)=o_{p}(1). \end{aligned}$$

(A.25)

To prove (A.24), we follow the steps of showing (A.20), we have for any $\tau >0$

$$\begin{aligned}&P\left\{ \sup \limits _{w\in \mathcal {W}}\left| \Vert H(w)S\Vert ^{2}-\text{tr}\{H'(w)H(w)\Sigma _{S}\}\right| / \check{R}_{n}(w)>\tau \right\} \\&\quad \le P\left\{ \sup \limits _{w\in \mathcal {W}}\displaystyle \sum _{t=1}^{K} \displaystyle \sum _{k=1}^{K}w_{t}w_{k} \left| S'H^{(t)}{'}H^{(k)}S-\text{tr}(H^{(t)}{'}H^{(k)} \Sigma _{S})\right|>\tau \check{\xi }_{n}\right\} \\&\quad \le \displaystyle \sum _{t=1}^{K} \displaystyle \sum _{k=1}^{K}P\left\{ \left| S'H'(w_{t}^{0}) H(w_{k}^{0})S-\text {tr}\{H'(w_{t}^{0})H(w_{k}^{0}) \Sigma _{S}\}\right| >\tau \check{\xi }_{n}\right\} \\&\quad \le \tau ^{-2N}\check{\xi }_{n}^{-2N} \displaystyle \sum _{t=1}^{K}\displaystyle \sum _{k=1}^{K}E \left[ S'H'(w_{t}^{0})H(w_{k}^{0})S-\text {tr}\{H'(w_{t}^{0}) H(w_{k}^{0})\Sigma _{S}\}\right] ^{2N} \\&\quad \le CC_{S}^{N}\tau ^{-2N}\check{\xi }_{n}^{-2N} \displaystyle \sum _{t=1}^{K}\displaystyle \sum _{k=1}^{K} \left[ \text {tr}\left\{ H'(w_{t}^{0})H(w_{k}^{0}) \Sigma _{S}H'(w_{k}^{0})H(w_{t}^{0})\right\} \right] ^{N} \\&\quad \le CC_{S}^{N}\tau ^{-2N}\check{\xi }_{n}^{-2N} \displaystyle \sum _{t=1}^{K}\displaystyle \sum _{k=1}^{K} \left[ \text {tr}\left\{ H(w_{k}^{0}) \Sigma _{S}H'(w_{k}^{0})\right\} \right] ^{N} \\&\quad \le CC_{S}^{N}\tau ^{-2N}\check{\xi }_{n}^{-2N}K \displaystyle \sum _{k=1}^{K}\left\{ \check{R}_{n} (w_{k}^{0})\right\} ^{N} =o_{p}(1). \end{aligned}$$

So (A.24) is satisfied. The fact $\text {tr}(H^{(t)}{'}H^{(k)})\le \left\{ \text {tr}(H^{(t)}{'}H^{(t)})\right\} ^{1/2} \left\{ \text {tr}(H^{(k)}{'}H^{(k)})\right\} ^{1/2}\le \sqrt{p_{t}p_{k}}$ together with conditions (C4) and (C6) demonstrate that

$$\begin{aligned}&\sup \limits _{w\in \mathcal {W}}\left| \text{tr}\{H'(w) H(w)\Sigma _{S}\}\right| /\check{R}_{n}(w)\\&\quad \le \sup \limits _{w\in \mathcal {W}} \left| \bar{\lambda }(\Sigma _{S})\text{tr} \left( \displaystyle \sum _{t=1}^{K}\displaystyle \sum _{k=1}^{K}w_{t}w_{k}H^{(t)}{'}H^{(k)}\right) \right| \Bigg /\check{\xi }_{n}\\&\quad \le C_{S}\cdot \bar{p}/\check{\xi }_{n}=o_{p}(1), \end{aligned}$$

which, along with (A.24), implies that $\sup \limits _{w\in \mathcal {W}}\Vert H(w)S\Vert ^{2}/\check{R}_{n}(w)=o_{p}(1)$. This together with Cauchy-Schwarz inequality and (A.19) indicate

$$\begin{aligned} \sup \limits _{w\in \mathcal {W}}\left| \mu _{0}'A'(w)H(w)S\right| / \check{R}_{n}(w)&\le \left\{ \Vert A(w)\mu _{0}\Vert ^{2}\Vert H(w)S\Vert ^{2}/\check{R}^{2}_{n}(w)\right\} ^{1/2}\\&\le \left\{ \Vert H(w)S\Vert ^{2}/\check{R}_{n}(w) \right\} ^{1/2}=o_{p}(1). \end{aligned}$$

Thus, we can get (A.25). By Lemma 2 and $\check{\xi }_{n}\rightarrow \infty$, it is not difficult to verify (A.23). Theorem 1 is then proved. $\square$

Proof of Theorem 2

When the true model is indeed linear, i.e. $\mu _{0i}=X_{i}'\beta$, we have $\hat{y}_{i}-\mu _{0i}=\zeta _{i}$, where $\zeta _{i}=\frac{\delta _{i}}{\pi _{i}}\epsilon _{i} +(1-\frac{\delta _{i}}{\hat{\pi }_{i}})X_{i}' (\hat{\beta }_{c}-\beta )+(\frac{1}{\hat{\pi }_{i}} -\frac{1}{\pi _{i}})\delta _{i}\epsilon _{i}$. Let $\epsilon =(\epsilon _{1}, \ldots , \epsilon _{n})'$ and $\zeta =(\zeta _{1}, \ldots , \zeta _{n})'$. From condition (C1) and the assumption that $\sigma _{i}^{2}$ is finite, we obtain that $\textbf{X}'\epsilon =O_{p}(n^{1/2})$, and hence $\textbf{X}'\zeta =O_{p}(n^{1/2})$. It is seen that

$$\begin{aligned} CV(w)&=\Vert \{I_{n}-H(w)\}\hat{Y}\Vert ^{2}+w'\Omega w \nonumber \\&=\Vert \zeta \Vert ^{2}+\{\bar{\beta }(w)-\beta \}'\textbf{X}' \textbf{X}\{\bar{\beta }(w)-\beta \}-2\zeta '\textbf{X} \{\bar{\beta }(w)-\beta \}+w'\Omega w, \end{aligned}$$

(A.26)

where $\Omega$ is a $K\times K$ matrix with the kjth element $\Omega _{kj}=\hat{Y}'(I_{n}-H^{(k)})'(T^{(k)}+T^{(j)}+T^{(k)} T^{(j)})(I_{n}-H^{(j)})\hat{Y}$. By conditions (C1) and (C2), it follows that $\Omega _{kj}=O_{p}(1)$. Hence, for any $w\in \mathcal {W}$,

$$\begin{aligned} w' \Omega w=O_{p}(1). \end{aligned}$$

(A.27)

Let k be the true model belonging to $\{1,\ldots , K\}$, then from Guo et al. (2017), we know that $\bar{\beta }^{(k)}-\beta =O_{p}(n^{-1/2})$, which together with (A.26) and (A.27) and condition (C1), implies $CV(w_{k}^{0})=\Vert \zeta \Vert ^{2}+\eta _{n}(w_{k}^{0})$ with

$$\begin{aligned} \eta _{n}(w_{k}^{0})=O_{p}(1). \end{aligned}$$

(A.28)

Therefore, $CV(\hat{w})\le CV(w_{k}^{0})=\Vert \zeta \Vert ^{2}+\eta _{n}(w_{k}^{0})$, which, together with (A.26), implies

$$\begin{aligned} \eta _{n}(w_{k}^{0})\ge \{\bar{\beta }(\hat{w})-\beta \}' \textbf{X}'\textbf{X}\{\bar{\beta }(\hat{w})-\beta \} -2\zeta '\textbf{X}\{\bar{\beta }(\hat{w})-\beta \} +\hat{w}'\Omega \hat{w}. \end{aligned}$$

(A.29)

Let $\Psi _{n}=n^{-1}\textbf{X}'\textbf{X}$. From (A.29), we obtain

$$\begin{aligned}&\underline{\lambda }(\Psi _{n})\Vert \sqrt{n}\{\bar{\beta }(\hat{w}) -\beta \}\Vert ^{2}\le \{\bar{\beta }(\hat{w})-\beta )\}\textbf{X}' \textbf{X}\{\bar{\beta }(\hat{w})-\beta \}\\&\quad \le \eta _{n}(w_{k}^{0})+2\zeta '\textbf{X} \{\bar{\beta }(\hat{w})-\beta \}-\hat{w}'\Omega \hat{w}\\&\quad \le \eta _{n}(w_{k}^{0})+2 \Vert n^{-1/2} \zeta '\textbf{X}\Vert \Vert \sqrt{n}\{\bar{\beta }(\hat{w})-\beta \}\Vert -\hat{w}'\Omega \hat{w}, \end{aligned}$$

and so

$$\begin{aligned}&\underline{\lambda }(\Psi _{n})\left[ \Vert \sqrt{n}\{\bar{\beta } (\hat{w})-\beta \}\Vert -\underline{\lambda }^{-1}(\Psi _{n}) \Vert n^{-1/2} \zeta '\textbf{X}\Vert \right] ^{2}\nonumber \\&\quad \le \eta _{n}(w_{k}^{0})+\{\underline{\lambda } (\Psi _{n})\}^{-1}\Vert n^{-1/2} \zeta {'}\textbf{X}\Vert ^{2} -\hat{w}'\Omega \hat{w}. \end{aligned}$$

(A.30)

Write $a_{n}=\eta _{n}(w_{k}^{0})+\{\underline{\lambda } (\Psi _{n})\}^{-1}\Vert n^{-1/2} \zeta {'}\textbf{X} \Vert ^{2}-\hat{w}'\Omega \hat{w},$ then (A.30) equals to

$$\begin{aligned} \Vert \sqrt{n}\{\bar{\beta }(\hat{w})-\beta \} \Vert&\in \Big [- \{\underline{\lambda }^{-1}(\Psi _{n})a_{n}\}^{1/2} +\underline{\lambda }^{-1}(\Psi _{n})\Vert n^{-1/2} \zeta {'} \textbf{X}\Vert {,} \\&\qquad \{\underline{\lambda }^{-1}(\Psi _{n})a_{n}\}^{1/2} +\underline{\lambda }^{-1}(\Psi _{n})\Vert n^{-1/2} \zeta {'}\textbf{X}\Vert \Big ], \end{aligned}$$

which, together with (A.27), (A.28) and $\textbf{X}'\zeta =O_{p}(n^{1/2})$, means $\sqrt{n}\{\bar{\beta }(\hat{w})-\beta \}=O_{p}(1)$. This completes the proof. $\square$