A Note on the Theme of Too Many Instruments*
| Date | 01 February 2009 |
| DOI | http://doi.org/10.1111/j.1468-0084.2008.00542.x |
| Author | David Roodman |
| Published date | 01 February 2009 |
135
©Center for Global Development 2009. Published by Blackwell Publishing Ltd, 9600 Garsington Road, Oxford OX4 2DQ, UK and
350 Main Street, Malden, MA 02148, USA.
OXFORD BULLETIN OF ECONOMICS AND STATISTICS, 71, 1 (2009) 0305-9049
doi: 10.1111/j.1468-0084.2008.00542.x
PRACTITIONERS’ CORNER
A Note on the Theme of Too Many InstrumentsÅ
David Roodman
Center for Global Development, Washington, DC, USA
(e-mail: droodman@cgdev.org)
Abstract
The difference and system generalized method of moments (GMM) estimators are
growing in popularity.As implemented in popular software, the estimators easily gen-
erate instruments that are numerous and, in system GMM, potentially suspect. Alarge
instrument collection overfits endogenous variables even as it weakens the Hansen
test of the instruments’ joint validity. This paper reviews the evidence on the effects
of instrument proliferation, and describes and simulates simple ways to control it.
It illustrates the dangers by replicating Forbes [American Economic Review (2000)
Vol. 90, pp. 869–887] on income inequality and Levine et al. [Journal of Monetary
Economics] (2000) Vol. 46, pp. 31–77] on financial sector development. Results in
both papers appear driven by previously undetected endogeneity.
Emperor Joseph II: My dear young man, don’t take it too hard. Your work is ingenious. It’s
quality work.And there are simply too many notes, that’s all. Just cut a few and it will be perfect.
Mozart: Which few did you have in mind, Majesty?
—Amadeus (1984)
I. Introduction
The concern at hand is not too many notes but too many instruments. If all econo-
metricians plied their craft with Mozart’s genius, the concern could be as humor-
ously dismissed. But we do not, so it must be taken seriously. Sargan, perhaps one
of the profession’s Mozarts, perceived the danger early on, in his paper introducing
the ‘Sargan test’:
*I thank Selvin Akkus for research assistance and Thorsten Beck, Decio Coviello, Kristin Forbes, Mead
Over, Jonathan Temple(editor) and two anonymous reviewers for helpful comments. I take sole responsibility
for all assertions and opinions expressed herein.
JEL Classification numbers: C23, G0, O40.
136 Bulletin
0
20
40
60
80
100
120
140
1991 1993 1995 1997 1999 2001 2003 2005
Arellano and
Bond (1991)
Blundell and
Bond (1998)
Source: Thomson Scientific, Social Sciences Citation
Figure 1. Citations of Arellano and Bond (1991) and Blundell and Bond (1998) per year, 1991–2006
A few calculations were made by the author on the order of magnitude of the errors
involved in this approximation. They were found to be proportional to the number of
instrumental variables, so that, if the asymptotic approximations are to be used, this
number must be small. (Sargan, 1958)
The popularity of the difference and system generalized method of moments
(GMM) estimators for dynamic panels has grown rapidly in recent years (Holtz-
Eakin, Newey and Rosen, 1988; Arellano and Bond, 1991;Arellano and Bover, 1995;
Blundell and Bond, 1998). (See Figure 1, which graphs citations by year for two of the
most important papers behind the estimators.) There are several reasons for this. The
estimators handle important modelling concerns – fixed effects and endogeneity of
regressors – while avoiding dynamic panel bias (Nickell, 1981). The flexible GMM
framework accommodates unbalanced panels and multiple endogenous variables.
Free software automates their use (Arellano and Bond, 1998; Doornik, Arellano and
Bond, 2002; Roodman, forthcoming).1But an underappreciated problem often arises
in the application of difference and system GMM: instrument proliferation.
The problem is not unique to these two estimators, and the general consequences
have been documented in the literature (Tauchen, 1986; Altonji and Segal, 1996;
Andersen and Sørensen, 1996; Ziliak, 1997; Bowsher, 2002). Textbooks even men-
tion in passing the poor performance of IV estimators when instruments are many
1Stata has included difference GMM functionality since version 7 and system GMM functionality since
version 10.
©Center for Global Development 2009
To continue reading
Request your trial