A Comparison of some non-parametric kernel estimators using simulations

Rabab Abdulrida Saleh; Iqbal Mahmood Alwan

doi:10.30526/37.4.3805

Authors

Rabab Abdulrida Saleh Department of Statistics, College of Administration and Economics, University of Baghdad, Baghdad, Iraq. https://orcid.org/0009-0004-8147-217X
Iqbal Mahmood Alwan Department of Statistics, College of Administration and Economics, University of Baghdad, Baghdad, Iraq. https://orcid.org/0000-0002-6656-7662

DOI:

https://doi.org/10.30526/37.4.3805

Keywords:

nonparametric regression, Nadaraya-Watson estimator, kernel functions, Quadratic, local polynomial estimator

Abstract

In this research, we conducted a comprehensive investigation of various kernel functions employed in the estimation of nonparametric regression functions. In particular, we investigated the Nadaraya-Watson method and local polynomial regression techniques involving linear, quadratic and cubic forms. These methods were evaluated using five different kernel functions: Gaussian, Epanechnikov, uniform, triangular and quartic, collectively referred to as GEUTQ.

The main objective of the work was to determine the best estimators for non-parametric kernel functions. To achieve this, we performed a rigorous comparison with simulation methods, different regression models and different sample sizes. The evaluation of the performance of the estimators was based on the mean absolute percentage error (AMAPE) assuming a standard normal distribution with a mean of zero and a variance of one.

Our simulation results and plots clearly show that the quadratic estimator (LP2) using the kernel function (G, E) consistently has the lowest (AMAPE) across all sample sizes and two models. Similarly, the local linear estimator (LP1) in the functions (U, T, Q) has the lowest (AMAPE) for all sample sizes and two models. As for the optimal functions, function (Q) is identified as the most effective kernel function among the options considered, leading to the lowest average values.

Furthermore, it is observed that as the sample size increases, the average values for the following methods decrease: Nadaraya, the linear method and the quadratic method. Conversely, the LP3 estimators, especially the linear cubic regression, turn out to be the least favorable and have relatively high values compared to the other estimators. These results provide valuable insights into the performance of different estimators and kernel functions in nonparametric regression models and thus contribute to future research and decision-making processes.

References

Speckman, P. Kernel smoothing in partial linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology 1988, 50(3), 413-436.

Hmood, M. Y.; Muslim, A. A comparison of some semiparametric estimators for consumption function regression. Journal of Economic and Administrative Science 2012, 12(67), 273-288. https://doi.org/10.33095/jeas.v18i67.1049.

Manzoor, M.A.; Akbar, A.; Ullahy, M.A. Performance of Nonparametric Regression Estimation with Diverse Covariates. Pakistan Journal of Social Sciences 2013, 33(1), 77-85.

Hmood, M. Y.; Mohamed, M.K. A comparison of the semiparametric estimators model using different smoothing methods. Journal of Economic and Administrative Sciences 2014, 20(75), 376-394.

Alwan, I. M.; Al-Doori, E.A. Hazard Rate Estimation Using Varying Kernel Function for Censored Data Type I. Baghdad Science Journal 2019, 16(3) (SUPPL.): 0793. http://dx.doi.org/10.21123/bsj.2019.16.3(Suppl.).0793.

Hassan, Y.A.; Hmood, M.Y. Estimation of return stock rate by using wavelet and kernel smoothers. Periodicals of Engineering and Natural Sciences 2020, 8(2), 602–612.

Ibrahim, M.K.; Al-Qazaz, Q.N.N. Comparison between the estimated of nonparametric methods by using the methodology of quantile regression models. Periodicals of Engineering and Natural Sciences 2020, 8(2), 572–579.

Habeeb, A.S.; Hmood, M.Y.; Mohammed, M.J. Estimation nonparametric autoregressive curve by smoothing spline method. International Journal of Agricultural and Statistical Sciences 2021, 17(2), 815-825. https://connectjournals.com/03899.2021.17.815.

Bhattacharyya, B.; Biswas, R.; Sujatha, K.; Chiphang, D.Y. Linear Regression Model to Study the Effects of Weather Variables on Crop Yield in Manipur State. International Journal of Agricultural and Statistical Sciences 2021, 17(1), 317-320.

Hall, P.; Presnel, B. Intentionally biased bootstrap methods. Journal of the Royal Statistical Society Series B: Statistical Methodology 1999, 61(1), 143-158. https://www.jstor.org/stable/2680742.

Liang, H. Estimation in partially linear models and numerical comparisons. Computational Statistics & Data Analysis 2006, 50(3), 675-687. https://doi.org/10.1016/j.csda.2004.10.007.

W. Härdle, Müller, M.; Sperlich, S.; Werwatz, A. Nonparametric and Semiparametric Models—An Introduction. Springer Verlag, Berlin, Heidelberg, Germany, 2005.

A.B. Tsybakov. Introduction to Nonparametric Estimation. Springer ScienceBusiness Media, LLC, New York, NY, USA, 2009.

Dinardo, J. Nonparametric density and regression estimation. Journal of Economic Perspectives 2001, 15(4), 11-28. https://www.jstor.org/stable/2696513.

Duin, R.P.W. On the choice of smoothing parameters of parent estimators of probability density functions. IEEE Transactions on Computers 1976, 1175-1179. http://dx.doi.org/10.1109/TC.1976.1674577.

Mousa, M.J.; Hmood, M.Y.; Nori, A.G. Comparison between VG-levy and kernel function estimation with application. Periodicals of Engineering and Natural Sciences 2021, 9(3), 106-112. https://doi.org/1021533/pen.v9i3.2079.g862.

Li, C. A Partially Linear Model with Its Applications. Communications in Statistics—Simulation and Computation 2013, 42(7), 1673-1680.

Loader, C. Local Regression and Likelihood. Springer-Verlag, New York, NY, USA, 1999.

Some, S.M.; Kokonendji, C.C. Effects of associated kernels in nonparametric multiple. University of Franche-Comté, Besançon, France 2014, 1-18. https://doi.org/101080/15598608.2016.1160010.

W. Härdle. Applied Nonparametric Regression Analysis. Cambridge University Press, Cambridge, UK, 1994.

Schindler, A. Bandwidth Selection in Nonparametric Kernel Estimation. Doctoral dissertation, Göttingen, Georg-August Universität, 2011.

Aydoğdu, H.; Kara, M. Nonparametric estimation in α-series processes. Computational Statistics and Data Analysis 2012, 56(1), 190-201.

Hansen, B.E. Lecture Notes on Nonparametrics. University of Wisconsin, Madison, WI, USA, 2009.

Fox, J. Introduction to Nonparametric Regression. McMaster University, 2005.

Fox, J. Nonparametric Regression. Dept. of Sociology – Canada, 2004,1-10.

Györfi, L.; Kohler, M.; Krzyżak, A.; Walk, H. A Distribution-Free Theory of Nonparametric Regression. Springer-Verlag, New York, NY, USA, 2002. https://doi.org/10.1007/b97848.

Hens, N. Non- and Semi-Parametric Techniques for Handling Missing Data. Limburgs Universitair Centrum (Belgium), 2005.

Jasim, O.R.; AL-Qazaz, O.N.N. Nonparametric estimation in doubly geometric stochastic processes. International Journal of Agricultural and Statistical Sciences 2021, 17(1), 2113–2122. https://connectjournals.com/03899.2021.17.

Takezawa, K. Introduction to Nonparametric Regression. John Wiley, Hoboken, NJ, USA, 2006, 90. https://doi.org/10.1007/s10182-006-0007-9.

Breidt, F.J.; Opsomer, J.D. Nonparametric and semiparametric estimation in complex surveys, in D. Pfeffermann, C.R. Rao (Eds.). Handbook of Statistics 2009, 29, 103-121. https://doi.org/10.1016/S0169-7161(09)00227-2.

Kromann, B. Nonparametric Kernel Density Estimation. Statistical Science, working paper, 1995.

Hollander, M.; Wolfe, D.A.; Chicken, E. Nonparametric Statistical Methods. John Wiley & Sons, 2013.

Avery, M. Literature Review for Local Polynomial Regression, Unpublished Manuscript, 2010.

Fan, J.; Gijbels, I. A study of variable bandwidth selection for local polynomial regression. Statistica Sinica 1996, 113-127. https://www.jstor.org/stable/24306002.

Fan, J.; Gijbels, I. Variable bandwidth linear regression smoothers. The Annals of Statistics 1992, 4, 2008-2036.

Fan, J.; Gijbels, I. Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. Journal of the Royal Statistical Society: Series B (Methodological) 1995, 57(2), 371-394. https://www.jstor.org/stable/24306002.

Fan, J. Local linear regression smoothers and their minimax efficiencies. The Annals of Statistics 1993, 21(1), 196-216. https://doi.org/10.1214/aos/1176349022.

Wand, M.P.; Jones, M.C. Kernel smoothing. Springer, 1995.

Hamilton, S.A.; Truong, Y.K. Local linear estimation in partly linear models. Journal of Multivariate Analysis 1997, 60(1), 1-19.

Wand, M.P.; Jones, M.C. Kernel Smoothing. Chapman and Hall, New York, NY, USA, 1995.

Mousa, M.J.; Hmood, M.Y.; Nori, A.G. Comparison between VG-levy and kernel function estimation with application. Periodicals of Engineering and Natural Sciences 2021, 9(3),40.

Mousa, M.J.; Hmood, M.Y.; Nori, A.G. Comparison between VG-levy and kernel function estimation with application. Periodicals of Engineering and Natural Sciences 2021, 9(3), 106-112. https://doi.org/1021533/pen.v9i3.2079.g862.

Ali, O.A. Suggesting multiphase regression model estimation with some threshold point. Journal of Mechanics of Continua and Mathematical Sciences 2020, 15(2), 212-223. https://doi.org/10.26782/jmcms.2020.02.00019.

Lmbens, G.; Kalyanaraman, K. Optimal bandwidth choice for the regression discontinuity estimator. Review of Economic Studies 2009, 79, 933–959.

Eubank, R.L.; Shucany, R.W. Adaptive bandwidth choice for kernel regression. Journal of Economic and Administrative Sciences 1990, 5, 1-15. https://doi.org/10.33095/jeas.v18i65.1112.

Habbema, J.D.F.; Hermans, J.; van den Broek, K. A stepwise discrimination analysis program using density estimation. Proceedings in Computational Statistics, Physica Verlag, Vienna, 1974.

Cao, R.; Cuevas, A.; Gonzalez-Manteiga, W. A comparative study of several smoothing methods in density estimation. Computational Statistics & Data Analysis 1994, 17(2), 153–176. https://doi.org/10.1016/0167-9473(92)00066-Z.

Scott, D.W.; Terrell, G.R. Biased and unbiased cross-validation in density estimation. Journal of the American Statistical Association 1987, 82, 1131–1146. https://doi.org/10.1080/01621459.1987.10478550.

Schafer, C.; Wasserman, L. Tutorial on Nonparametric Inference with R. Carnegie Mellon University, working paper, 2013.