Survey participation effects in conflict research
Author | Carlo Koos,Alexander De Juan |
DOI | 10.1177/0022343320971034 |
Published date | 01 July 2021 |
Date | 01 July 2021 |
Articles
Survey participation effects in conflict
research
Alexander De Juan
Institute for Social Sciences, University of Osnabru
¨ck
Carlo Koos
Chr. Michelsen Institute
Abstract
Do survey participants in conflict zones respond differently if they have been interviewed before? Academic and
policy interest in postwar political opinion has increased tremendously. One unexpected consequence of this surge of
survey research is a growing probability that individuals will be interviewed multiple times. However, if participating
in one survey causes respondents to change their attitudes or behavior, their subsequent survey responses may be
biased in comparison to the rest of the sample population. Our article aims to investigate such ‘survey participation
effects’ in conflict contexts. We draw on original survey data collected in the eastern Democratic Republic of Congo
(DRC). In our representative sample, 18% of respondents report that they have been interviewed before. Multi-
variate analyses demonstrate that their stated attitudes on social relations, political institutions, gender norms, and
wartime victimization differ substantively from the responses of first-time interviewees. Moreover, our analyses
indicate that experienced respondents have specific response styles – in particular, a tendency to support extreme
response options. While substantive bias in multivariate analyses seems to be rather rare, our findings indicate that
researchers should be aware of the footprints of data collection efforts in areas frequently targeted by household and
opinion surveys.
Keywords
conflict, conditioning, data collection, methods, survey
Introduction
An increasing number of research teams have been travel-
ing to conflict zones to collect data for aid impact evalua-
tions, academic research projects or more general opinion
polls. This development comes with an increasing prob-
ability that households and individuals will be surveyed
multiple times. However, being exposed to interview
situations mayaffect people’s attitudes, behavior, and gen-
eral response behavior. This, in turn, can influence their
answers in subsequent surveys. This article aims to inves-
tigate the potential effects of repeated interview participa-
tion on survey-based descriptive and inferential analyses
focusing on the micro-level dynamics of violent conflict.
It has been recognized since at least 1940 that a ‘big
problem yet unsolved is whether repeated interviews are
likely, in themselves, to influence a respondent’s opi-
nions’ (Lazarsfeld, 1940: 128). For example, being
exposed to a survey question might make respondents
reflect and gather information about the topic after the
survey. Repeated interviews may also increase trust in
survey processes and the willingness to provide sensitive
information to unknown enumerators. Conversely, par-
ticipation in multiple surveys may create a certain degree
of survey fatigue that reduces people’s motivation to
substantively engage with survey items (Lund, Shackle-
ton & Luckert, 2011).
Corresponding author:
alexander.dejuan@uni-osnabrueck.de
Journal of Peace Research
2021, Vol. 58(4) 623–639
ªThe Author(s) 2021
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/0022343320971034
journals.sagepub.com/home/jpr
All these factors could affect people’s opinions and
corresponding answers in subsequent interviews. More-
over, they may also shape people’s response styles: ‘a sys-
tematic tendency to respond to a range of questionnaire
items on some other basis than the specific item content’
(Paulhus, 1991: 17). A large body of research in market-
ing, cognitive psychology, and survey methodology has
investigated the potential effects of repeated survey partic-
ipation – focusing almost exclusively on participation in
panel surveys in Western states (see comprehensive
reviews and discussions in Cantor, 2007; Warren &
Halpern-Manners, 2012). Several studies find evidence
of ‘panel conditioning’ effects as outlined above, albeit
of different types and in different directions (Bergmann
& Barth, 2018; Warren & Halpern-Manners, 2012). At
the same time, a substantial number of studies have found
no or only very small effects (Smith, Gerber & Orlich,
2003; Bartels, 1999). This inconclusiveness may at least
partly be traced back to the heterogeneous effects of mul-
tiple survey participation depending on survey design,
content, and socio-economic contexts.
We investigate these survey participation effects in face-
to-face perception surveys in the context of armed
conflicts. The effects of survey experience may be partic-
ularly pronounced under these specific circumstances: in
the light of pressing socio-economic needs and little for-
mal education, individuals may have less crystallized atti-
tudes on abstract concepts and less practice in translating
their opinions into choices in closed-ended questions.
High levels of violence and polarization can also increase
distrust in unknown interviewers and reluctance towards
disclosing sensitive information. Finally, as a result of a
high dependence on international assistance, respondents
may initially place high (and potentially unwarranted)
expectations on survey participation when interviews are
considered part of foreign aid planning.
Under these scope conditions, previous survey partici-
pation may have particularly strong effects in terms of (a)
increasing respondents’ practice in thinking about and
responding to survey items, (b) decreasing distrust in the
interview process, and (c) creating a certain degree of dis-
illusionment and survey fatigue. Consequently, the
responsepatterns of experiencedinterviewees may substan-
tially differ from those of first-time survey participants.
We investigate these potential effects using original
survey data collected in the eastern Democratic Republic
of the Congo (DRC). Together with a Congolese survey
organization, we collected data from 1,000 randomly
sampled households in 100 villages in the eastern prov-
ince of South Kivu in 2017. Importantly, we also asked
whether respondents had previously participated in
similar surveys. Our ana lyses demonstrate substantive
correlations of this self-reported prior survey participa-
tion with a wide array of outcomes relating to people’s
social and political attitudes. Moreover, we document
significant associations with response styles: people who
have already participated in surveys tend to select more
extreme response options in Likert-scaled items. Lastly,
we show that previous survey experience can moderately
bias certain ‘standard’ analyses of the sociopolitical legacy
of violence. This, however, seems to be the case only for
a limited and specific set of models.
These findings contribute to research on the specific
methodological challenges of empirical analyses in
conflict-affected areas (e.g. Bru
¨ck et al., 2016; Haer &
Becher, 2012). Previous studies have highlighted that
interview situations and interviewer–interviewee interac-
tions differ from those in surveys conducted in stable
political contexts. Among others, particularities of con-
flict zones can lead to more pronounced interviewer
effects that shape the interviewees’ response patterns
(Adida et al., 2016). We demonstrate that surveys in
conflict zones also risk being affected by previous survey
participation. While substantive bias in multivariate
analyses seems to be moderate, our findings indicate that
researchers should be aware of the footprints of data
collection efforts in areas frequently targeted by house-
hold and opinion surveys.
Opinion surveys in armed conflicts
Research on the causes and consequences of violent con-
flict has long focused on the country level, investigating
how political, social, and economic factors increase or
decrease the risk of violent conflict, its intensity or its
termination. In recent years, however, the field has expe-
rienced a micro-level turn. A growing number of
researchers are now focusing on trying to understand the
motives, attitudes, and interpretations of individuals
actively involved in or passively affected by violent con-
flict. Many of these studies rely on survey data collected
in standardized perception interviews (see overview in
Bru
¨ck et al., 2016).
In order to get a general sense of this research field we
have collected information on research articles published
in the past ten years (2008–17) featuring survey-based
analyses of conflict zones (a total of 113 articles).
1
The
1
We have conducted systematic keyword-based searches in JSTOR
and Google Scholar and have used information provided in three
recent review articles (Bauer et al., 2016; Bru
¨ck et al., 2016;
Zu
¨rcher, 2017). We have used the following combinations of
624 journal of PEACE RESEARCH 58(4)
To continue reading
Request your trial