Uncovering Regimes in Out of Sample Forecast Errors from Predictive Regressions*
| Published date | 01 June 2021 |
| Author | Anibal Emiliano Da Silva Neto,Jesús Gonzalo,Jean‐Yves Pitarakis |
| Date | 01 June 2021 |
| DOI | http://doi.org/10.1111/obes.12418 |
Uncovering Regimes in Out of Sample Forecast
Errors from Predictive Regressions*
ANIBAL EMILIANO DASILVA NETO,†JES
´
US GONZALO‡and
JEAN-YVES PITARAKIS†
*Department of Economics, University of Southampton, Southampton, UK (e-mail: A.Emiliano-
Da-Silva-Neto@soton.ac.uk; j.pitarakis@soton.ac.uk)
†Department of Economics, Universidad Carlos III de Madrid, Getafe, Madrid, Spain (e-mail:
jesus.gonzalo@uc3m.es)
Abstract
We introduce a set of test statistics for assessing the presence of regimes in out of
sample forecast errors produced by recursively estimated linear predictive regressions
that can accommodate multiple highly persistent predictors. Our test statistics are
designed to be robust to the chosen starting window size and are shown to be both
consistent and locally powerful. Their limiting null distributions are also free of
nuisance parameters and hence robust to the degree of persistence of the predictors.
Our methods are subsequently applied to the predictability of the value premium
whose dynamics are shown to be characterized by state dependence.
I. Introduction
A vast body of recent empirical research documented the presence of state dependence
in the forecast errors produced by models used to generate forecasts of a broad range
of economic and financial variables such as stock and bond returns, commodity
returns, rates of inflation, currency returns among many others. State dependence in
this context takes the form of forecast errors having different quality characteristics
such as lower variances in periods of economic recessions versus expansions. In Golez
and Koudijs (2018) for instance the authors considered century long stock market data
and documented the considerable strengthening of the in-sample and out of sample
predictive power of dividend yields for stock returns during recessions. Chauvet and
Potter (2013) remarked that predictability of output growth is much harder during
JEL Classification numbers:C12, C22. C53, C58.
*We wish to thank the Editor and two anonymous referees for valuable comments and suggestions. We also
thank seminar participants at the Zaragoza Time Series Workshop 2018, Bilgi University Workshop on
Advances in Econometric Methods 2018, Southampton Econometrics Workshop 2018, University of Cologne,
University of Essex and University of Nottingham; Gonzalo gratefully acknowledges financial support from the
Spanish Ministerio de Economia y Competitividad (grants ECO2016-78652, PID2019-104960GB-I00 and Maria
de Maeztu MDM 2014-0431) and MadEco-CM (grant S205/HUM-3444). Pitarakis thanks the British Academy
for financial support through grant SRG170220.
713
©2021 The Department of Economics, University of Oxford and John Wiley & Sons Ltd.
OXFORD BULLETIN OF ECONOMICS AND STATISTICS, 83, 3 (2021) 0305-9049
doi: 10.1111/obes.12418
recessions while Gargano, Pettenuzzo and Timmermann (2019) established that
commodity returns are predictable using macroeconomic information but solely during
recessions.
This state dependence in the behaviour of forecast errors has been typically
documented through a descriptive comparison of prediction errors (e.g. lower MSEs
during recessions than expansions) or the use of recession dummies within the
underlying forecasting models. Numerous papers concerned with the predictability of
the equity premium with valuation ratios documented important differences in out of
sample goodness of fit metrics across NBER business cycle dates (see Rapach, Strauss
and Zhou, 2010; Li and Tsiakas, 2017 amongst others).
The main goal of this paper was to introduce formal diagnostic tools for explicitly
testing for the presence of broadly defined regimes in the out-of-sample prediction
errors generated from predictive regression models. We are interested in both the levels
of forecast errors and their squares as considering the two series can convey useful
information on both misspecification issues and regime specificity in MSEs. Rather
than thinking of regimes as matching business cycle dates we take a broader view of
the notion of state dependence and associate regimes with observed proxies of the state
of the economy exceeding or falling below particular levels. Our proposed methods
require solely the computation of recursive least squares residuals which are then used
within a CUSUM type construct and are therefore very easily implementable. Our
operating framework is also flexible enough to accommodate predictive regressions
with multiple highly persistent predictors of possibly different persistence strengths.
Suppose for instance that one wishes to evaluate the predictability of the equity
premium with the commonly used Goyal and Welch predictors (Welch and Goyal,
2008; Goyal and Welch, 2014). These include quantities such as dividend yields,
price-to-earnings ratios, interest rates all known to be highly persistent variables with
potentially different degrees of persistence and typically modelled as nearly integrated
processes with a nuisance parameter that parameterizes persistence strength. How does
one go about formally testing whether forecasts generated from such models lead to
forecast errors that behave differently across the business cycle?
The issue is of great practical importance as the presence of regime specificity in
prediction errors would call for a reassessment of the models used to generate forecasts
and in particular motivate a switch to nonlinear specifications that are explicitly able to
capture episodic predictability as for instance in Gonzalo and Pitarakis (2012, 2017)
where the authors considered the inclusion of threshold effects within predictive
regressions driven by a single highly persistent predictor. Such piecewise linear
structures are particularly convenient as they allow the forecaster to control the
particular indicator used for proxying economic times or more generally sentiment. As
such they are not necessarily restricted to a rigid regime structure dictated by formal
externally provided business cycle dates. We view the testing procedures introduced in
this paper as useful practical diagnostic tools that can be used to motivate the explicit
inclusion of regime dependence within the predictive model itself. Although
postdiagnostic re-evaluation issues are beyond the scope of this paper such
specifications have been shown to lead to considerable gains in prediction accuracy as
demonstrated in an in-sample and single predictor based equity premium forecasting
©2021 The Department of Economics, University of Oxford and John Wiley & Sons Ltd
714 Bulletin
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your 3-day Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your 3-day Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your 3-day Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your 3-day Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your 3-day Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting