Qualitative investigation of theoretical models: the value of process tracing
| Author | M Taylor Fravel,Jack Paine,Peter Lorentzen |
| Date | 01 July 2017 |
| Published date | 01 July 2017 |
| DOI | 10.1177/0951629816664420 |
| Subject Matter | Articles |
Article
Journal of Theoretical Politics
2017, Vol.29(3) 467–491
ÓThe Author(s) 2016
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0951629816664420
journals.sagepub.com/home/jtp
Qualitative investigation of
theoretical models: the value
of process tracing
Peter Lorentzen
University of California,Berkeley, USA
M Taylor Fravel
Massachusetts Instituteof Technology,USA
Jack Paine
University of Rochester, USA
Abstract
Political scientists frequently use qualitative evidence to support or evaluate the empirical applic-
ability of formal models. Despite this widespread practice, neither the qualitative methods litera-
ture nor research on empirically evaluating formal models systematically address the topic. This
article makes three contributionsto bridge this gap. First, it demonstrates that formal models and
qualitative evidence are indeed frequently combined in current research. Second, it shows how
process tracing can be as important a tool for empirically assessing models as statistical testing,
because models and process tracing share a common focus on understanding causal mechanisms.
Last, it provides new guidelines for using process tracing that focus on issues specific to the mod-
eling enterprise, illustrated with examplesfrom recent research.
Keywords
Comparative politics; empiricalimplications of theoretical models; game theory; international rela-
tions; process tracing
Corresponding author:
Peter Lorentzen,University of California, Berkeley,210 Barrows Hall, Berkeley,C A 94720-1950,USA.
Email: lorentzen@berkeley.edu
1. Introduction
Since the introduction of formal models into political science, their relationship
with the rest of the discipline has provoked vigorous discussion. Early debates
about whether formalization or rational choice approaches contribute to our
understanding of politics have largely given way to more applied discussions about
the appropriate relationship between formal theories and empirical research.
1
Yet,
these discussions have centered on how (or even whether) formal models should be
tested with statistical methods, providing little guidance for scholars interested in
assessing formal theories with qualitative evidence. Likewise, although the litera-
ture on qualitative methods has seen a strong resurgence of activity in recent years,
it does not address the particular issues that arise when seeking to use qualitative
methods to evaluate formal theories.
This methodological gap is perplexing, because in practice articles introducing a
new formal model frequently include qualitative evidence. We surveyed all such
articles published from 2006 to 2013 on topics in international relations and com-
parative politics from a set of prominent journals and found that qualitative evi-
dence appeared in over half the articles that empirically assessed the model.
Furthermore, recent critiques of prominent models of audience costs and of regime
change have sparked vigorous debates about core arguments in these subfields.
2
Yet despite its common use, formal theorists often dismiss the value of qualita-
tive evidence. Coming from the other direction, scholars using qualitative evidence
to evaluate existing formal models do not express these qualms, but modelers often
question the conclusions drawn by these studies. The lack of agreed-upon stan-
dards can lead scholars from different methodological backgrounds to talk past
each other, hindering scientific progress. The applied formal modeling enterprise is
now at an unsatisfying middle ground whereby qualitative evidence is routinely
used to convince readers of the value of a model, yet the evidence is not presented
or evaluated rigorously because it is qualitative.
The frequency with which scholars from diverse methodological backgrounds
use qualitative evidence to evaluate formal models, along with a lack of clear meth-
odological guidance, suggests two options: develop better standards or abandon
the practice. We argue for developing better standards. In particular, formal mod-
els and the increasingly prominent qualitative method known as process tracing
share an under-recognized affinity—a focus on causal mechanisms—that makes
process tracing a valuable tool for the empirical investigation of formal models.
Qualitative evidence that strongly supports or contradicts a model’s causal pro-
cesses can help to either strengthen or to weaken the model’s claims to empirical
applicability. Such process tracing can be as convincing as statistical tests.
A sharp need exists for systematic standards on using process tracing to evalu-
ate formal models. In this article, we take a first step toward such standards, focus-
ing on issues that are unique to or are particularly important for the modeling
endeavor rather than surveying process tracing in general. We address (1) how to
use process tracing to evaluate the causal mechanisms embodied in a formal model,
(2) what types of evidence to collect to conduct process tracing, and (3) how to
468 Journal of Theoretical Politics 29(3)
To continue reading
Request your trial