Clements, Michael P. and Harvey, David Ian (2006) Forecast encompassing tests and probability forecasts. Working Paper. Coventry: University of Warwick, Department of Economics. (Warwick economic research papers.

Preview

PDF WRAP_Clements_twerp_774.pdf - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader Download (429Kb)

Abstract

We consider tests of forecast encompassing for probability forecasts, for both quadratic and logarithmic scoring rules. We propose test statistics for the null of forecast encompassing, present the limiting distributions of the test statistics, and investigate the impact of estimating the forecasting models’ parameters on these distributions. The small-sample performance of the various statistics is investigated, both in terms of small numbers of forecasts and model estimation sample sizes. Two empirical applications show the usefulness of the tests for the evaluation of recession probability forecasts from logit models with different leading indicators as explanatory variables, and for evaluating survey-based probability forecasts.

Item Type:	Working or Discussion Paper (Working Paper)
Subjects:	H Social Sciences > HB Economic Theory
Divisions:	Faculty of Social Sciences > Economics
Library of Congress Subject Headings (LCSH):	Economic forecasting, Statistical hypothesis testing, Recessions, Business cycles, Economics -- Simulation methods
Series Name:	Warwick economic research papers
Publisher:	University of Warwick, Department of Economics
Place of Publication:	Coventry
Date:	21 July 2006
Number:	No.774
Number of Pages:	36
Status:	Not Peer Reviewed
Access rights to Published version:	Open Access
References:	Anderson, H. M., and Vahid, F. (2001). Predicting the probability of a recession with nonlinear autoregressive leading indicator models. Macroeconomic Dynamics, 5, 482—505. Birchenhall, C. R., Jessen, H., Osborn, D., and Simpson, P. (1999). Predicting U.S. business cycle regimes. Journal of Business & Economic statistics, 17, No. 3, (July), 313—323. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 75, 1—3. Carruth, A. A., Hooker, M. A., and Oswald, A. J. (1998). Unemployment equilibria and input prices: Theory and evidence from the United States. Review of Economics and Statistics, 80, 621—628. Chauvet, C., and Potter, S. (2002). Predicting a recession: evidence from the yield curve in the presence of structural breaks. Economic Letters, 77, 245—253. Chong, Y. Y., and Hendry, D. F. (1986). Econometric evaluation of linear macro-economic models. Review of Economic Studies, 53, 671—690. Reprinted in Granger, C. W. J. (ed.) (1990), Modelling Economic Series. Oxford: Clarendon Press. Clark, T. E., and McCracken, M. W. (2001). Tests of equal forecast accuracy and encompassing for nested models. Journal of Econometrics, 105, 85—110. Clemen, R. T. (1989). Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5, 559—583. Reprinted in Mills, T. C. (ed.) (1999), Economic Forecasting. The International Library of Critical Writings in Economics. Cheltenham: Edward Elgar. Clemen, R. T., and Winkler, R. L. (1999). Combining probability distributions from experts in risk analysis. Risk Analysis, 19, 187—203. Croushore, D., and Stark, T. (2001). A real-time data set for macroeconomists. Journal of Econometrics, 105(November), 111—130. Dawid, A. P. (1986). Probability forecasting. In Kotz, S., Johnson, N. L., and Read, C. B. (eds.), Encyclopedia of Statistical Sciences, vol. 7, pp. 210—218: John Wiley and Sons. Diebold, F. X., and Lopez, J. A. (1996). Forecast evaluation and combination. In Maddala, G. S., and Rao, C. R. (eds.), Handbook of Statistics, Vol. 14, pp. 241—268: Amsterdam: North-Holland. Diebold, F. X., and Mariano, R. S. (1995). Comparing predictive accuracy. Journal of Business and Economic Statistics, 13, 253—263. Reprinted in Mills, T. C. (ed.) (1999), Economic Forecasting. The International Library of Critical Writings in Economics. Cheltenham: Edward Elgar. Diebold, F. X., and Rudebusch, G. D. (1989). Scoring the leading indicators. Journal of Business, 62, 369—391. Ericsson, N. R. (1993). Comment on ‘On the limitations of comparing mean squared forecast errors’, by M.P. Clements and D.F. Hendry. Journal of Forecasting, 12, 644—651. Estrella, A., and Mishkin, F. S. (1998). Predicting US recessions: Financial variables as leading indicators. Review of Economics and Statistics, 80, 45—61. Fair, R. C., and Shiller, R. J. (1990). Comparing information in forecasts from econometric models. American Economic Review, 80, 39—50. Faust, J., Rogers, J. H., and Wright, J. H. (2003). Exchange rate forecasting: The errors we’ve really made. Journal of International Economic Review, 60, 35—39. Faust, J., Rogers, J. H., and Wright, J. H. (2005). News and noise in G-7 GDP announcements. Journal of Money, Credit and Banking, 37 (3), 403—417. Genest, C., and Zidek, J. V. (1986). Combining probability distributions: A critique and an annotated bibliography. Statistical Science, 1, 114—148. Good, I. (1952). Rational decisions. Journal of the Royal Statistical Society. Series B, 14 (No. 1), 107—114. Granger, C.W. J., and Newbold, P. (1973). Some comments on the evaluation of economic forecasts. Applied Economics, 5, 35—47. Reprinted in Mills, T. C. (ed.) (1999), Economic Forecasting. The International Library of Critical Writings in Economics. Cheltenham: Edward Elgar. Granger, C. W. J., and Pesaran, M. H. (2000). Economic and statistical measures of forecast accuracy. Journal of Forecasting, 19, 537—560. Hamilton, J. D. (1983). Oil and the Macroeconomy since World War II. Journal of Political Economy, 91, 228—248. Hamilton, J. D. (1996). This is what happened to the oil price-macroeconomy relationship. Journal of Monetary Economics, 38, 215—220. Hamilton, J. D. (2000). What is an Oil Shock?. Journal of Econometrics, 113, 363—398. Hamilton, J. D., and Kim, D. H. (2000). A re-examination of the predictability of economic activity using the yield spread. NBER Working Papers, 7954. Harvey, D. I., Leybourne, S., and Newbold, P. (1998). Tests for forecast encompassing. Journal of Business and Economic Statistics, 16, 254—259. Reprinted in Mills, T. C. (ed.) (1999), Economic Forecasting. The International Library of Critical Writings in Economics. Cheltenham: Edward Elgar. Hooker, M. A. (1996). Whatever happened to the oil price-macroeconomy relationship?. Journal of Monetary Economics, 38, 195—213. Kamstra, M., and Kennedy, P. (1998). Combining qualitative forecasts using logit. International Journal of Forecasting, 14, 83—93. Koenig, E. F., Dolmas, S., and Piger, J. (2003). The use and abuse of real-time data in economic forecasting. The Review of Economics and Statistics, 85(3), 618—628. Lee, K., Ni, S., and Ratti, A. (1995). Oil shocks and the macroeconomy: The role of price variability. Energy Journal, 16, 39—56. Mork, K. A. (1989). Oil and the Macroeconomy when prices go up and down: An extension of Hamilton’s results. Journal of Political Economy, 97, 740—744. Nelson, C. R. (1972). The prediction performance of the FRB-MIT-PENN model of the US economy. American Economic Review, 62, 902—917. Reprinted in Mills, T. C. (ed.) (1999), Economic Forecasting. The International Library of Critical Writings in Economics. Cheltenham: Edward Elgar. Newbold, P., and Harvey, D. I. (2002). Forecasting combination and encompassing. In Clements, M. P., and Hendry, D. F. (eds.), A Companion to Economic Forecasting, pp. 268—283: Oxford: Blackwells. West, K. D. (1996). Asymptotic inference about predictive ability. Econometrica, 64, 1067—1084. West, K. D. (2001). Tests for forecast encompassing when forecasts depend on estimated regression parameters. Journal of Business and Economic Statistics, 19, 29—33. West, K. D. (2006). Forecasting evaluation. In Elliott, G., Granger, C., and Timmermann, A. (eds.), Handbook of Economic Forecasting, Volume 1. Handbook of Economics 24, pp. 99—134: Elsevier, Horth-Holland. West, K. D., and McCracken, M. W. (1998). Regression-based tests of predictive ability. International Economic Review, 39, 817—840. Winkler, R. L. (1996). Scoring rules and the evaluation of probabilities (with discussion). Test, 5 (No. 1), 1—60.
URI:	http://wrap.warwick.ac.uk/id/eprint/1425

Request changes to a record

Actions (login required)

View Item