The Library

A generic algorithm for reducing bias in parametric estimation

Tools

Kosmidis, Ioannis and Firth, David. (2010) A generic algorithm for reducing bias in parametric estimation. Electronic Journal of Statistics, Vol.4 . pp. 1097-1112. ISSN 1935-7524

Preview

PDF
WRAP_Firth_kosmidis-firth-2010-ejs.pdf - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (448Kb)

Official URL: http://dx.doi.org/10.1214/10-EJS579

Abstract

A general iterative algorithm is developed for the computation of reduced-bias parameter estimates in regular statistical models through adjustments to the score function. The algorithm unifies and provides appealing new interpretation for iterative methods that have been published previously for some specific model classes. The new algorithm can usefully be viewed as a series of iterative bias corrections, thus facilitating the adjusted score approach to bias reduction in any model for which the first- order bias of the maximum likelihood estimator has already been derived. The method is tested by application to a logit-linear multiple regression model with beta-distributed responses; the results confirm the effectiveness of the new algorithm, and also reveal some important errors in the existing literature on beta regression.

Item Type:	Journal Article
Subjects:	Q Science > QA Mathematics
Divisions:	Faculty of Science > Statistics
Library of Congress Subject Headings (LCSH):	Parameter estimation, Mathematical models
Journal or Publication Title:	Electronic Journal of Statistics
Publisher:	Institute of Mathematical Statistics
ISSN:	1935-7524
Date:	2010
Volume:	Vol.4
Page Range:	pp. 1097-1112
Identification Number:	10.1214/10-EJS579
Status:	Peer Reviewed
Access rights to Published version:	Open Access
Funder:	Engineering and Physical Sciences Research Council (EPSRC)
References:	[1] Albert, A. and J. Anderson (1984). On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71(1), 1–10. MR0738319 [2] Breslow, N. E. and X. Lin (1995). Bias correction in generalised linear mixed models with a single component of dispersion. Biometrika 82, 81–91. MR1332840 [3] Bull, S. B., C. Mak, and C. Greenwood (2002). A modified score function estimator for multinomial logistic regression in small samples. Com- putational Statistics and Data Analysis 39, 57–74. MR1895558 [4] Cook, R. D., C.-L. Tsai, and B. C. Wei (1986). Bias in nonlinear regression. Biometrika 73, 615–623. MR0897853 [5] Cordeiro, G. and M. Toyama Udo (2008). Bias correction in generalized nonlinear models with dispersion covariates. Communications in Statistics: Theory and Methods 37(14), 2219–225. MR2526676 [6] Cordeiro, G. M. and P. McCullagh (1991). Bias correction in generalized linear models. Journal of the Royal Statistical Society, Series B: Methodological 53(3), 629–643. MR1125720 [7] Cordeiro, G. M. and K. L. P. Vasconcellos (1997). Bias correction for a class of multivariate nonlinear regression models. Statistics & Probability Letters 35, 155–164. MR1483269 [8] Cox, D. R. and D. V. Hinkley (1974). Theoretical Statistics. London: Chapman & Hall Ltd. MR0370837 [9] Cox, D. R. and E. J. Snell (1968). A general definition of residuals (with discussion). Journal of the Royal Statistical Society, Series B: Methodolog- ical 30, 248–275. MR0237052 [10] Cribari-Neto, F. and A. Zeileis (2010). Beta regression in R. Journal of Statistical Software 34 (2), 1–24. [11] Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency) (with discussion). The Annals of Statistics 3, 1189–1217. MR0428531 [12] Ferrari, S. and F. Cribari-Neto (2004). Beta regression for modelling rates and proportions. Journal of Applied Statistics 31(7), 799–815. MR2095753 [13] Firth, D. (1992). Bias reduction, the Jeffreys prior and GLIM. In L. Fahrmeir, B. Francis, R. Gilchrist, and G. Tutz (Eds.), Advances in GLIM and Statistical Modelling: Proceedings of the GLIM 92 Conference, Munich, New York, pp. 91–100. Springer. [14] Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika 80(1), 27–38. MR1225212 [15] Gart, J. J., H. M. Pettigrew, and D. G. Thomas (1985). The effect of bias, variance estimation, skewness and kurtosis of the empirical logit on weighted least squares analyses. Biometrika 72, 179–190. [16] Heinze, G. and M. Schemper (2002). A solution to the problem of separation in logistic regression. Statistics in Medicine 21, 2409–2419. [17] Heinze, G. and M. Schemper (2004). A solution to the problem of monotone likelihood in Cox regression. Biometrics 57, 114–119. MR1833296 [18] Kosmidis, I. (2009). On iterative adjustment of responses for the reduction of bias in binary regression models. Technical Report 09-36, CRiSM working paper series. [19] Kosmidis, I. and D. Firth (2009). Bias reduction in exponential family nonlinear models. Biometrika 96(4), 793–804. MR2564491 [20] Lin, X. and N. E. Breslow (1996). Bias correction in generalized linear mixed models with multiple components of dispersion. Journal of the American Statistical Association 91, 1007–1016. MR1424603 [21] McCullagh, P. (1987). Tensor Methods in Statistics. London: Chapman and Hall. MR0907286 [22] Mehrabi, Y. and J. N. S. Matthews (1995). Likelihood-based methods for bias reduction in limiting dilution assays. Biometrics 51, 1543–1549. [23] Ospina, R., F. Cribari-Neto, and K. L. Vasconcellos (2006). Improved point and interval estimation for a beta regression model. Compu- tational Statistics and Data Analysis 51(2), 960 – 981. MR2297500 [24] Pace, L. and A. Salvan (1997). Principles of Statistical Inference: From a Neo-Fisherian Perspective. London: World Scientific. MR1476674 [25] Pettitt, A. N., J. M. Kelly, and J. T. Gao (1998). Bias correction for censored data with exponential lifetimes. Statistica Sinica 8, 941–964. MR1651517 [26] Prater, N. H. (1956). Estimate gasoline yields from crudes. Petroleum Refiner 35, 236–238. [27] R Development Core Team (2008). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0. [28] Sartori, N. (2006). Bias prevention of maximum likelihood estimates for scalar skew normal and skew t distributions. Journal of Statistical Planning and Inference 136, 4259–4275. MR2323415 [29] Schaefer, R. L. (1983). Bias correction in maximum likelihood logistic regression. Statistics in Medicine 2, 71–78. [30] Simas, A. B., W. Barreto-Souza, and A. V. Rocha (2010). Improved estimators for a general class of beta regression models. Computational Statistics and Data Analysis 54(2), 348–366. [31] Smithson, M. and J. Verkuilen (2006). A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychological Methods 11(1), 54–71. [32] Zorn, C. (2005). A solution to separation in binary response models. Political Analysis 13, 157–170.
URI:	http://wrap.warwick.ac.uk/id/eprint/4341