Skip to main content
SpringerLink
Log in

On Sufficient Conditions for the Consistency of Local Linear Kernel Estimators

  • Research Articles
  • Published:
Mathematical Notes Aims and scope Submit manuscript

Abstract

The consistency of classical local linear kernel estimators in nonparametric regression is proved under constraints on design elements (regressors) weaker than those known earlier. The obtained conditions are universal with respect to the stochastic nature of design, which may be both fixed regular and random and is not required to consist of independent or weakly dependent random variables. Sufficient conditions for pointwise and uniform consistency of classical local linear estimators are stated in terms of the asymptotic behavior of the number of design elements in certain neighborhoods of points in the domain of the regression function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In the Russian-language literature, the term “deterministic design” is often used.

  2. This means that, with probability \(1\), \(c_1h\leq \lim\sup_{ t\in[0,1]}\#(N_{n,h}(t))/n\leq c_2h\), where the constants \(c_1\) and \(c_2\) do not depend on \(t\) and \(h\).

  3. This means that \({\mathbb P}\bigl(\#(N_{n,h}(t))/(nh)\in [c_1,c_2]\bigr)\to 1\), where \(c_1>0\) and \(c_2>0\).

References

  1. J. Fan and I. Gijbels, Local Polynomial Modelling and Its Applications, in Monographs on Stat. Appl. Prob. (Chapman & Hall, London, 1996), Vol. 66.

    MATH  Google Scholar 

  2. W. Hardle, M. Muller, S. Sperlich, and A. Werwatz, Nonparametric and Semiparametric Models, in Springer Ser. in Statistics (Springer- Verlag, Berlin, 2004).

    Book  MATH  Google Scholar 

  3. J. Fan and Q. Yao, Nonlinear Time Series. Nonparametric and Parametric Methods, in Springer Ser. in Statistics (Springer- Verlag, New York, 2003).

    Book  MATH  Google Scholar 

  4. L. Györfi, M. Kohler, A. Krzyzak, and H. Walk, A Distribution-Free Theory of Nonparametric Regression, in Springer Ser. in Statistics (Springer- Verlag, New York, 2002).

    Book  MATH  Google Scholar 

  5. W. Härdle, Applied Nonparametric Regression, in Econometric Soc. Monog. (Cambridge Univ. Press, Cambridge, 1990), Vol. 19.

    Book  MATH  Google Scholar 

  6. C. Loader, Local Regression and Likelihood, in Statistics and Computing (Springer- Verlag, New York, 1999).

    Book  MATH  Google Scholar 

  7. H.-G. Müller, Nonparametric Regression Analysis of Longitudinal Data, in Lecture Notes in Stat. (Springer- Verlag, New York, 1988), Vol. 46.

    MATH  Google Scholar 

  8. I. S. Borisov, Yu. Yu. Linke, and P. S. Ruzankin, “Universal weighted kernel-type estimators for some class of regression models,” Metrika 84 (2), 141–166 (2021).

    Article  MathSciNet  MATH  Google Scholar 

  9. Y. Linke, I. Borisov, P. Ruzankin, V. Kutsenko, E. Yarovaya, and S. Shalnova, “Universal local linear kernel estimators in nonparametric regression,” Mathematics 10 (15), 2693 (2022).

    Article  Google Scholar 

  10. Yu. Yu. Linke and I. S. Borisov, “Insensitivity of Nadaraya–Watson estimators to design correlation,” Comm. Statist. Theory Methods 51 (19), 6909–6918 (2022).

    Article  MathSciNet  MATH  Google Scholar 

  11. Yu. Yu. Linke, “Towards insensitivity of Nadaraya–Watson estimators with respect to design correlation,” Teor. Veroyatnost. Primenen. 68 (2), 236–252 (2023).

    MATH  Google Scholar 

  12. J. Beran and Y. Feng, “Local polynomial estimation with a FARIMA-GARCH error process,” Bernoulli 7 (5), 733–550 (2001).

    Article  MathSciNet  MATH  Google Scholar 

  13. D. Benelmadani, K. Benhenni, and S. Louhichi, “Trapezoidal rule and sampling designs for the nonparametric estimation of the regression function in models with correlated errors,” Statistics 54 (1), 59–96 (2020).

    Article  MathSciNet  MATH  Google Scholar 

  14. X. Tang, M. Xi, Y. Wu, and X. Wang, “Asymptotic normality of a wavelet estimator for asymptotically negatively associated errors,” Statist. Probab. Lett. 140, 191–201 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  15. W. Gu, G. G. Roussas, and L. T. Tran, “On the convergence rate of fixed design regression estimators for negatively associated random variables,” Statist. Probab. Lett. 77 (12), 1214–1224 (2007).

    Article  MathSciNet  MATH  Google Scholar 

  16. J. S. Wu and C. K. Chu, “Nonparametric estimation of a regression function with dependent observations,” Stochastic Process. Appl. 50 (1), 149–160 (1994).

    Article  MathSciNet  MATH  Google Scholar 

  17. K. Benhenni, S. Hedli-Griche, and M. Rachdi, “Estimation of the regression operator from functional fixed-design with correlated errors,” J. Multivariate Anal. 101 (2), 476–490 (2010).

    Article  MathSciNet  MATH  Google Scholar 

  18. D. A. Ioannides, “Consistent nonparametric regression: some generalizations in the fixed design case,” J. Nonparametr. Statist. 2 (3), 203–213 (1993).

    Article  MathSciNet  MATH  Google Scholar 

  19. A. A. Georgiev, “Asymptotic properties of the multivariate Nadaraya–Watson regression function estimate: the fixed design case,” Statist. Probab. Lett. 7 (1), 35–40 (1989).

    Article  MathSciNet  MATH  Google Scholar 

  20. W. Hardle and S. Luckhaus, “Uniform consistency of a class of regression function estimators,” Ann. Statist. 12 (2), 612–623 (2012).

    MathSciNet  MATH  Google Scholar 

  21. L. P. Devroye, “The uniform convergence of the Nadaraya–Watson regression function estimate,” Canad. J. Statist. 6 (2), 179–191 (1979).

    Article  MathSciNet  MATH  Google Scholar 

  22. É. A. Nadaraya, “Remarks on non-parametric estimates of density functions and regression curves,” Theory Probab. Appl. 15 (1), 139–142 (1970).

    Article  MATH  Google Scholar 

  23. H. Liero, “Strong uniform consistency of nonparametric regression function estimates,” Probab. Theory Related Fields 82 (4), 587–614 (1989).

    Article  MathSciNet  MATH  Google Scholar 

  24. Y. P. Mack and B. W. Silvermann, “Weak and strong uniform consistency of kernel regression estimates,” Z. Wahrsch. Verw. Gebiete 61 (3), 405–415 (1982).

    Article  MathSciNet  MATH  Google Scholar 

  25. U. Einmahl and D. M. Mason, “Uniform in bandwidth consistency of kernel-type function estimators,” Ann. Statist. 33 (3), 1380–1403 (2005).

    Article  MathSciNet  MATH  Google Scholar 

  26. T. Gasser and J. Engel, “The choice of weights in kernel regression estimation,” Biometrica 77 (2), 377–381 (1990).

    Article  MathSciNet  MATH  Google Scholar 

  27. C. K. Chu and W.-S. Deng, “An interpolation method for adapting to sparse design in multivariate nonparametric regression,” J. Statist. Plann. Inference 116 (1), 91–111 (2003).

    Article  MathSciNet  MATH  Google Scholar 

  28. Q. Li, X. Lu, and A. Ullah, “Multivariate local polynomial regression for estimating average derivatives,” J. Nonparametr. Stat. 15 (4–5), 607–624 (2003).

    Article  MathSciNet  MATH  Google Scholar 

  29. J. Gu, Q. Li, and J.-C. Yang, “Multivariate local polynomial kernel estimators: leading bias and asymptotic distribution,” Econometric Rev. 34 (6–10), 978–1009 (2015).

    MathSciNet  MATH  Google Scholar 

  30. O. B. Linton and D. T. Jacho-Chavez, “On internally corrected and symmetrized kernel estimators for nonparametric regression,” TEST 19 (1), 166–186 (2010).

    Article  MathSciNet  MATH  Google Scholar 

  31. R. Kulik and P. Lorek, “Some results on random design regression with long memory errors and predictors,” J. Statist. Plann. Inference 141 (1), 508–523 (2011).

    Article  MathSciNet  MATH  Google Scholar 

  32. J. Shen and Y. Xie, “Strong consistency of the internal estimator of nonparametric regression with dependent data,” Statist. Probab. Lett. 83 (8), 1915–1925 (2013).

    Article  MathSciNet  MATH  Google Scholar 

  33. E. Masry, “Multivariate local polynomial regression for time series: uniform strong consistency and rates,” J. Time Ser. Anal. 17 (6), 571–599 (1996).

    Article  MathSciNet  MATH  Google Scholar 

  34. X. Li, W. Yang, and S. Hu, “Uniform convergence of estimator for nonparametric regression with dependent data,” J. Inequal. Appl., 142 (2016).

    Article  MathSciNet  MATH  Google Scholar 

  35. S. J. Hong and O. B. Linton, “Asymptotic properties of a Nadaraya–Watson type estimator for regression functions of infinite order,” SSRN Electronic Journal (2016).

    Google Scholar 

  36. G. G. Roussas, “Nonparametric regression estimation under mixing conditions,” Stochastic Process. Appl. 36 (1), 107–116 (1990).

    Article  MathSciNet  MATH  Google Scholar 

  37. E. Masry, “Nonparametric regression estimation for dependent functional data: asymptotic normality,” Stochastic Process. Appl. 115 (1), 155–177 (2005).

    Article  MathSciNet  MATH  Google Scholar 

  38. J. Jiang and Y. P. Mack, “Robust local polynomial regression for dependent data,” Statist. Sinica 11 (3), 705–722 (2001).

    MathSciNet  MATH  Google Scholar 

  39. E. Masry, “Local linear regression estimation under long-range dependence: strong consistency and rates,” IEEE Trans. Inform. Theory 47 (7), 2863–2875 (2001).

    Article  MathSciNet  MATH  Google Scholar 

  40. N. V. Millionshchikov, “Asymptotic normality of regression estimates for weakly dependent random fields,” Moscow Univ. Math. Bull. 60 (2), 1–6 (2005).

    MathSciNet  Google Scholar 

  41. B. E. Hansen, “Uniform convergence rates for kernel estimation with dependent data,” Econometric Theory 24 (3), 726–748 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  42. J. Gao, S. Kanaya, D. Li, and D. Tjostheim, “Uniform consistency for nonparametric estimators in null recurrent time series,” Econometric Theory 31 (5), 911–952 (2015).

    Article  MathSciNet  MATH  Google Scholar 

  43. Q. Wang and N. Chan, “Uniform convergence rates for a class of martingales with application in non-linear cointegrating regression cointegrating regression,” Bernoulli 20 (1), 207–230 (2014).

    Article  MathSciNet  MATH  Google Scholar 

  44. N. Chan and Q. Wang, “Uniform convergence for nonparametric estimators with nonstationary data,” Econometric Theory 30 (5), 1110–1133 (2014).

    Article  MathSciNet  MATH  Google Scholar 

  45. O. Linton and Q. Wang, “Nonparametric transformation regression with nonstationary data,” Econometric Theory 32 (1), 1–29 (2016).

    Article  MathSciNet  MATH  Google Scholar 

  46. Q. Wang and P. C. B. Phillips, “Structural nonparametric cointegrating regression,” Econometrica 77 (6), 1901–1948 (2009).

    Article  MathSciNet  MATH  Google Scholar 

  47. H. A. Karlsen, T. Myklebust, and D. Tjostheim, “Nonparametric estimation in a nonlinear cointegration type model,” Ann. Statist. 35 (1), 252–299 (2007).

    Article  MathSciNet  MATH  Google Scholar 

  48. J. Chen, J. Gao, and D. Li, “Estimation in semi-parametric regression with non-stationary regressors,” Bernoulli 18 (2), 678–702 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  49. N. N. Chentsov, “Weak convergence of stochastic processes whose trajectories have no discontinuities of the second kind and the heuristic approach to the Kolmogorov–Smirnov tests,” Theory Probab. Appl. 1, 140–144 (1956).

    Article  Google Scholar 

  50. Yu. Yu. Linke, “Asymptotic properties of one-step \(M\)-estimators,” Comm. Statist. Theory Methods 48 (16), 4096–4118 (2019).

    Article  MathSciNet  MATH  Google Scholar 

  51. Yu. Yu. Linke and I. S. Borisov, “Constructing explicit estimators in nonlinear regression problems,” Theory Probab. Appl. 63 (1), 22–44 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  52. Yu. Yu. Linke and I. S. Borisov, “Constructing initial estimators in one-step estimation procedures of nonlinear regression,” Statist. Probab. Lett. 120, 87–94 (2017).

    Article  MathSciNet  MATH  Google Scholar 

  53. F. Yao, “Asymptotic distributions of nonparametric regression estimators for longitudinal or functional data,” J. Multivariate Anal. 98 (1), 40–56 (2007).

    Article  MathSciNet  MATH  Google Scholar 

  54. L. R. Cheruiyot, “Local linear regression estimator on the boundary correction in nonparametric regression estimation,” J. Stat. Theory Appl. 19 (3), 460–471 (2020).

    Article  Google Scholar 

  55. Yu. Yu. Linke and I. S. Borisov, “Universal nonparametric kernel estimates for mean functions and the covariance of random processes,” Teor. Veroyatnost. Primenen. (in press).

    Google Scholar 

  56. P. Qiu, Image Processing and Jump Regression Analysis, in Wiley Ser. Probab. and Stat. (Wiley, Hoboken, NJ, 2005).

    Book  MATH  Google Scholar 

  57. E. Rio, “Moment inequalities for sums of dependent random variables under projective conditions,” J. Theoret. Probab. 22 (1), 146–163 (2009).

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The author thanks the referee for the careful reading of the manuscript as well as for comments and remarks.

Funding

This work was supported by the Program of fundamental scientific studies of the Siberian Branch of the Russian Academy of Sciences (grant no. FWNF-2022-0015).

Author information

Authors and Affiliations

  1. Sobolev Institute of Mathematics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, 630090, Russia

    Yu. Yu. Linke

Authors
  1. Yu. Yu. Linke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu. Yu. Linke.

Additional information

Translated from Matematicheskie Zametki, 2023, Vol. 114, pp. 353–369 https://doi.org/10.4213/mzm13906.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Linke, Y.Y. On Sufficient Conditions for the Consistency of Local Linear Kernel Estimators. Math Notes 114, 308–321 (2023). https://doi.org/10.1134/S0001434623090043

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0001434623090043

Keywords

Search

Navigation