Using Military Spending Data: The Complexity of Simple Inference
| DOI | 10.1177/0022343399036006005 |
| Published date | 01 November 1999 |
| Author | James H. Lebovic |
| Date | 01 November 1999 |
| Subject Matter | Articles |
Using Military Spending Data: The Complexity of
Simple Inference
JAMES H. LEBOVIC
Department of Political Science, George Washington University
This study assesses the reliability of estimates of the direction of military spending growth obtained from
two main sources – the US Arms Control and Disarmament Agency (ACDA) and Stockholm
International Peace Research Institute (SIPRI). It examines the average directional agreement between
early and late ACDA and SIPRI spending estimates for a full sample of countries and for seven different
regions (Africa, East Asia, Latin America, Middle East, South Asia, NATO Europe, and the Warsaw
Pact). It shows that the direction of ACDA and SIPRI estimates diverge significantly over time and that
the two data sources appear especially challenged when estimating the sign of smaller, and especially
negative, growth-rate changes and of spending in regions (Africa and the Middle East) where growth-
rates vary markedly. It further establishes that, when a single source publishes consistent directional
estimates, these estimates can diverge considerably from those published by the other source. Based on
the findings, this study proposes a set of simple validation procedures and tests their strengths and
weaknesses on various sets of countries.
Introduction
Users of foreign military spending data are
twice challenged: they must account for per-
plexing global armament patterns with scant
and misleading information. Although they
are fully aware of the problems of generating
useful models, extrapolating from trends,
and forecasting the future, they have failed
to recognize the problems of generating esti-
mates for the recent and distant past. This is
a significant failing since military spending
data contain large amounts of error and bias
when judged by the degree to which esti-
mates for the same countries and years
change over time and differ by source
(Lebovic, 1995, 1998). By this standard,
error and bias in military growth-rate esti-
mates are severe and might even exceed the
magnitude of estimated growth (Lebovic,
1998). When using inaccurate and mis-
leading measures, data users can confuse
error and bias with actual spending changes
and patterns. They will therefore fail to
adopt appropriate coping strategies.
The presence of bias and error has
obvious implications for those who regard
military spending as an indicator of govern-
ment intent. Military spending figures that
are too high or too low create a false
* I wish to thank Lee Sigelman, Paul Wahlbeck, Michael
Brzoska, and the referees from JPR for their helpful com-
ments and advice. I especially want to thank Langche
Zeng, with her notepad, for convincing me that ‘great
ideas’ are sometimes bad math. Information on the data
used in this article can be obtained by contacting the
author at Lebovic@gwu.edu. The data can be obtained
from: http://gwis2.circ.gwu.edu/~lebovic. The analysis in
this study was performed using a SAS mainframe com-
puter package.
681
journal of
peace
R
ESEARCH
© 1999 Journal of Peace Research,
vol. 36, no. 6, 1999, pp. 681–697
Sage Publications (London, Thousand
Oaks, CA and New Delhi)
[0022-3433(199911)36:6; 681–697; 011132]
at SAGE Publications on December 7, 2012jpr.sagepub.comDownloaded from
impression of a government’s commitment
to armament or disarmament. Incorrect data
also challenge the planning of governments
when they assess their military preparedness
against a baseline of adversary military
spending. The presence of bias and error
confound academic researchers too. Because
military spending data is the subject of
important scholarly research, and because
research findings vary with the data
employed and the countries and time
periods examined, the possibility is strong
that what we know – or what we think we
know – about a diverse set of topics has been
influenced by problematic data. For
example, the fact that error may outweigh
meaningful variation in the data could
explain the mixed support found for ‘arms-
race models’ in which a country’s military
spending is portrayed as a response to the
spending of another country (for US–Soviet
applications, see Hollist, 1977; Ward,
1984). Moreover, error could contribute to
the frequent finding in budget trade-off
research that military budgets do not grow at
the expense of civilian budgets (Eichenberg,
1992), as well as providing mixed support
for the hypothesis that military spending
inhibits economic growth (Chan, 1995).1
Most certainly, problems of error challenge
researchers who delve back into historical
periods for which information is murky and
incomplete to derive military expenditure
data. For example, military expenditures are
employed as an indicator of national capa-
bility in the Correlates of War data used by
analysts to help explain whether and how
wars are fought (Sample, 1998; Wallace,
1990). While the obvious challenges of
working with historical evidence can instill
an awareness of the fragile informational
underpinnings upon which conclusions
often rest, the possibility exists in all of the
above research that data inaccuracies cause
analysts to see relationships where they do
not exist or to accept incorrectly the null
hypothesis of no relationship between vari-
ables (known in statistics as Type II and I
errors, respectively).
What can data users do then? One poss-
ible response to measurement problems is to
treat the data as a blunt tool by reducing
levels of measurement. More specifically,
users can employ figures to make directional
judgements, asking whether annual military
spending has increased or declined. Policy
analysts often resort to ordinal measurement
and, in particular, directional indicators to
make simple points about global change.
Illustrating this is the practice found in year-
end accounts (published by many research
organizations) of the changing state of the
world on matters of public concern – popu-
lation, industrialization, poverty and malnu-
trition, environmental pollution, and
military spending. In such reports, analysts
frequently highlight how a given phenom-
enon has changed by emphasizing both its
direction of growth and the consistency of its
direction of growth. Note, for instance, that
the chapter on global military expenditures
in the most recent yearbook published by
the Stockholm International Peace Research
Institute (SIPRI, 1998: 185) starts with the
sentence, ‘World military expenditure is still
declining but the rate of decline is slowing
down’. To acknowledge that the growth in
military spending is declining and that the
rate of decline is slowing down is to use two
ordinal measures, one being the direction of
change, to communicate the main properties
of the trend.2Indeed, the reputable organiz-
ations that distribute military spending data
journal of PEACE RESEARCH volume 36 / number 6 / november 1999682
1Admittedly, error in an arms-race model need not com-
promise research since governments presumably respond
to erroneous and correct data alike.
2The first of these ordinal measures appears to have three
values – amounts have increased, decreased, or remained
the same. The second has many more values – the rate of
decline is increasing, decreasing, or unchanged; the rate of
increase is increasing, decreasing, or unchanged; spending
remains constant, spending has reversed its decline, and so
on.
at SAGE Publications on December 7, 2012jpr.sagepub.comDownloaded from
To continue reading
Request your trial