How to make elite experiments work in International Relations
| Author | Haley J. Swedlund,Heidi Hardt,Simone Dietrich |
| Date | 01 June 2021 |
| DOI | 10.1177/1354066120987891 |
| Published date | 01 June 2021 |
E
JR
I
https://doi.org/10.1177/1354066120987891
European Journal of
International Relations
2021, Vol. 27(2) 596 –621
© The Author(s) 2021
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1354066120987891
journals.sagepub.com/home/ejt
How to make elite
experiments work in
International Relations
Simone Dietrich
University of Geneva, Switzerland
Heidi Hardt
University of California, Irvine, USA
Haley J. Swedlund
Radboud University, Netherlands
Abstract
For decades, many International Relations (IR) scholars did not engage in elite
experiments, because they viewed it as too risky, too costly, or too difficult to implement.
However, as part of a behavioral turn in IR, a growing number of scholars have begun to
adopt the method in their own research. This shift raises important questions. Under
what conditions do elite experiments add value to IR scholarship? How can scholars
overcome the logistical and ethical challenges of sampling such an elusive group? This
article makes an original conceptual contribution to methodological debates on the
role of behavioral approaches by analyzing experiments on foreign policy elites. We
analyze the method’s strengths and weaknesses, evaluate ethical considerations, and
present what is—to the best of our knowledge—the most comprehensive set of
implementation guidelines. Our article draws on recently published IR research and
argues that the payoffs from elite experiments are well worth the effort.
Keywords
Elite experiments, methodology, pluralism, foreign policy, experiments, International
Relations
Corresponding author:
Haley J. Swedlund, Radboud University, Heyendaalseweg 141, Nijmegen, 6525 AJ, Netherlands.
Email: h.swedlund@fm.ru.nl
987891EJT0010.1177/1354066120987891European Journal of International RelationsDietrich et al.
research-article2021
Article
Dietrich et al. 597
Introduction
Scholars in International Relations (IR) increasingly employ experiments on foreign
policy elites despite the challenges associated with implementation (e.g. Mintz et al.,
2006; Bethke (2016); Findley et al 2017a, 2017b; Hardt, 2018a, 2018b, Swedlund,
2017a; Hafner-Burton et al., 2017; Saunders, 2018; Yarhi-Milo et al., 2018; Busby
et al., 2019; Renshon 2015; Tomz et al., 2020; Dietrich, 2021). Previous studies have
criticized the method as too costly, too difficult, or simply too risky to carry out (Hafner-
Burton et al., 2013; Peabody, 1990). Thus, most scholarship comprising IR’s growing
body of experimental research—that is, IR’s “behavioral revolution” (Davis and
McDermott, 2020; IO, 2017; ISR, 2007)1—uses non-elite samples like students, the
public, and Amazon Mechanical Turk (i.e. Redd, 2002; Kertzer, 2017; Findley et al.,
2013; Bayram and Holmes, 2019). However, findings from elite experiments should be
better able to approximate elite behavior; research suggests that non-elite experiments
do not consistently predict elites’ actual decisions and actions (e.g. Druckman and Kam,
2011). Given that elite experiments do have certain advantages, when is the method
worth the costs?
In this article, we identify a set of conditions under which elite experiments add
value to IR research, and we introduce what we believe to be the first set of practical
guidelines for implementation. We offer IR scholars explicit guidance on navigating
upfront investments, ethically accessing and recruiting a sufficiently large sample of
elites, and deploying the experiment at minimum cost. Drawing on successful and ethi-
cally conducted examples, we posit that elite experiments elicit new and important find-
ings due to the markedly different ways that elites, relative to non-elites, behave,
including how they calculate risks, make decisions, and respond to incentives (Mintz
et al., 2006; Tetlock, 2005).
We employ Hafner-Burton et al.’s (2013: 369) definition of elites as core decision-
makers who (a) occupy “top positions in social and political structures,” (b) “have the
highest indices in their branch of authority,” and (c) “exercise significant influence over
social and political change” (also Pareto, 1935; Pakulski, 2008). Examples of elites
include individuals occupying high-level positions in states and international institutions
(e.g. members of legislatures, senior judges, ambassadors, military representatives in
foreign and defense ministries, assistant secretaries general) and individuals operating at
the highest levels (e.g. presidents, prime ministers and secretaries general).
Existing methodological scholarship has not yet specified when elite experiments can
help scholars address their research questions. Related work has assessed the value and
logistics of other types of experiments (e.g. cognitive neuroscience, natural experiments,
etc.) (see typology in Mintz et al., 2011; McDermott, 2002a, 2002b, 2011; Gerber and
Green, 2012). Additionally, several leading journals have devoted special issues to
exploring experimental research in IR (IO, 2017; ISQ, 2011). However, this body of
work has not investigated elite experiments as a unique type of experimental research.
Therefore, this article makes a conceptual contribution to existing scholarship on
methodological pluralism in IR by providing a comprehensive presentation of practical
advice for employing elite experiments as a methodology. We strongly support methodo-
logical openness and encourage the use of elite experiments as complementary to tradi-
tional methods. With respect to scope, we focus mainly on interview and survey-based
experiments, excluding field experiments (e.g. those involving randomized evaluations
To continue reading
Request your trial