Ley, Eduardo and Steel, Mark F. J. (2010) Mixtures of g-priors for Bayesian model averaging with economic applications. Working Paper. Coventry: University of Warwick. Centre for Research in Statistical Methodology. (Working papers, Vol.2010).

Preview

PDF WRAP_Ley_10-23w.pdf - Published Version - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader Download (705Kb)

Abstract

We examine the issue of variable selection in linear regression modeling, where we have a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the ap- propriate subset. Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. Our main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. We combine a Binomial-Beta prior on model size with a g-prior on the coefficients of each model. In addition, we assign a hyperprior to g, as the choice of g has been found to have a large impact on the results. For the prior on g, we examine the Zellner-Siow prior and a class of Beta shrinkage priors, which covers most choices in the recent literature. We propose a benchmark Beta prior, inspired by earlier findings with fixed g, and show it leads to consistent model selection. Inference is conducted through a Markov chain Monte Carlo sampler over model space and g. We examine the performance of the various priors in the context of simulated and real data. For the latter, we consider two important applications in economics, namely cross-country growth regression and returns to schooling. Recommendations to applied users are provided.

Item Type:	Working or Discussion Paper (Working Paper)
Subjects:	Q Science > QA Mathematics
Divisions:	Faculty of Science > Statistics
Library of Congress Subject Headings (LCSH):	Regression analysis, Bayesian statistical decision theory
Series Name:	Working papers
Publisher:	University of Warwick. Centre for Research in Statistical Methodology
Place of Publication:	Coventry
Date:	2010
Volume:	Vol.2010
Number:	No.23
Number of Pages:	25
Status:	Not Peer Reviewed
Access rights to Published version:	Open Access
References:	Andrews, D.R., and C.L. Mallows (1974) \Scale Mixtures of Normal Distributions," Jour- nal of the Royal Statistical Society, B, 36: 99{102. Berger, J.O. (1985) Statistical Decision Theory and Bayesian Analysis, 2nd ed., New York: Springer. Bernardo, J.M., and A.F.M. Smith (1994) Bayesian Theory, Chichester: John Wiley. Bottolo L., and Richardson S. (2008), \Fully Bayesian Variable Selection Using g-Priors," Working paper, Imperial College. Bottolo L., and Richardson S. (2010), \Evolutionary Stochastic Search for Bayesian Model Exploration," Bayesian Analysis, 5: 583{618. Brock, W., S. Durlauf and K. West (2003) \Policy Evaluation in Uncertain Economic Environments," (with discussion) Brookings Papers of Economic Activity, 1: 235{322. Brown, P.J., M. Vannucci and T. Fearn (1998) \Bayesian Wavelength Selection in Multi- component Analysis," Journal of Chemometrics, 12: 173{182. Carvalho, C.M., N.G. Polson and J.G. Scott (2010) \The Horseshoe Estimator for Sparse Signals," Biometrika, 97: 465{480. Chib, S. (1995) \Marginal Likelihood from the Gibbs Output," Journal of the American Statistical Association, 90: 1313{1321. Ciccone, A. and Jarocinski, M. (2010). Determinants of Economic Growth: Will Data Tell? American Economic Journal: Macroeconomics, 2: 222{246. Clyde, M.A., and E.I. George (2004) \Model Uncertainty," Statistical Science, 19: 81{94. Clyde, M.A., J. Ghosh and M. Littman (2010) \Bayesian adaptive sampling for variable selection and model averaging", Journal of Computational and Graphical Statistics, forthcoming. Cui, W., and George, E. I. (2008) \Empirical Bayes vs. fully Bayes variable selection," Journal of Statistical Planning and Inference, 138: 888{900. Eicher, T.S., C. Papageorgiou and A.E. Raftery (2010) \Determining Growth Determi- nants: Default Priors and Predictive Performance in Bayesian Model Averaging, With Applications to Growth Determinants," Journal of Applied Econometrics, forthcoming. Feldkircher, M. and S. Zeugner (2009) \Benchmark Priors Revisited: On Adaptive Shrink- age and the Supermodel Effect in Bayesian Model Averaging," IMF Working Paper 09/202. Feldkircher, M. and S. Zeugner (2010) \The Impact of Data Revisions on the Robustness of Growth Determinants: A Note on `Determinants of Economic Growth. Will Data Tell' ?," Working Paper 2010-12, University of Salzburg. Fernandez, C., E. Ley and M.F.J. Steel (2001a) \Benchmark Priors for Bayesian Model Averaging," Journal of Econometrics, 100: 381{427. Fernandez, C., E. Ley and M.F.J. Steel (2001b) \Model Uncertainty in Cross-Country Growth Regressions," Journal of Applied Econometrics, 16: 563{76. Fernandez, C., and M.F.J. Steel (2000) \Bayesian Regression Analysis With Scale Mixtures of Normals," Econometric Theory, 16: 80-101 Foster, D.P., and E.I. George (1994), \The Risk In ation Criterion for multiple regression," Annals of Statistics, 22: 1947{1975. Forte, A., M.J. Bayarri, J.O. Berger and G. Garca-Donato (2010), \Closed-form objective Bayes factors for variable selection in linear models", poster presentation Frontiers of Statistical Decision Making and Bayesian Analysis in honour of Jim Berger. Gneiting, T. and A.E. Raftery (2007) "Strictly Proper Scoring Rules, Prediction and Es- timation," Journal of the American Statistical Association, 102: 359{378. Gradshteyn, I.S. and I.M. Ryzhik (1994) Table of Integrals, Series and Products, San Diego: Academic Press, 5th ed. Hoeting, J.A., D. Madigan, A.E. Raftery and C.T. Volinsky (1999) \Bayesian model av- eraging: A tutorial," Statistical Science 14: 382{401. Jeffreys, H. (1961) Theory of Probability, Oxford: Clarendon Press, 3rd ed. Kass, R.E. and L. Wasserman (1995) \A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion," Journal of the American Statistical Associa- tion, 90: 928{934. Liang, F., R. Paulo, G. Molina, M.A. Clyde, and J.O. Berger (2008) \Mixtures of g-priors for Bayesian Variable Selection," Journal of the American Statistical Association, 103: 410{423. Ley, E. and M.F.J. Steel (2009) \On the Effect of Prior Assumptions in Bayesian Model Averaging With Applications to Growth Regression," Journal of Applied Econometrics, 24: 651{674. Madigan, D. and J. York (1995) \Bayesian graphical models for discrete data," Interna- tional Statistical Review, 63: 215{232. Maruyama, Y. and E.I. George (2010) \gBF: A Fully Bayes Factor with a Generalized g-prior," Technical Report, University of Pennsylvania, available at http://arxiv.org/abs/0801.4410 Mitchell, T.J. and J.J. Beauchamp (1988) \Bayesian variable selection in linear regression (with discussion), " Journal of the American Statistical Association, 83: 1023{1036. Nott, D.J. and R. Kohn (2005) \Adaptive Sampling for Bayesian Variable Selection," Biometrika, 92: 747{763 Raftery, A.E., D. Madigan, and J. A. Hoeting (1997) \Bayesian Model Averaging for Linear Regression Models," Journal of the American Statistical Association, 92: 179{191. Sala-i-Martin, X.X., G. Doppelhofer and R.I. Miller (2004) \Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, 94: 813{835. Strawderman, W. (1971) \Proper Bayes minimax estimators of the multivariate normal mean," Annals of Statistics, 42: 385{388. Tobias, J.L. and Li, M. (2004) \Returns to Schooling and Bayesian Model Averaging; A Union of Two Literatures," Journal of Economic Surveys, 18: 153{180. Verdinelli, I. and Wasserman, L. (1995) \Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio," Journal of the American Statistical Association, 90: 614-618. Zellner, A. (1971) An Introduction to Bayesian Inference in Econometrics, New York: Wiley. Zellner, A. (1986) \On assessing prior distributions and Bayesian regression analysis with g- prior distributions," in Bayesian Inference and Decision Techniques: Essays in Honour of Bruno de Finetti, eds. P.K. Goel and A. Zellner, Amsterdam: North-Holland, pp. 233{243. Zellner, A. and Siow, A. (1980) \Posterior odds ratios for selected regression hypothe- ses," (with discussion) in Bayesian Statistics, eds. J.M. Bernardo, M.H. DeGroot, D.V. Lindley and A.F.M. Smith, Valencia: University Press, pp. 585{603.
URI:	http://wrap.warwick.ac.uk/id/eprint/35121

Request changes to a record

Actions (login required)

View Item