Book Review: Evidence-based Policy: A Realist Perspective
| Date | 01 March 2007 |
| Published date | 01 March 2007 |
| DOI | 10.1177/1035719X0700700110 |
| Subject Matter | Book Review |
BOOK REVIEWS
Evaluation Journal of Australasia, Vol. 7, No. 1, 2007
56
Title: Evidence-based Policy: A Realist Perspective
Author: Ray Pawson
Publisher: Sage, London
Publication date: 2006
Extent/type:196 pages, paperback
Price: A$66 from Footprint Books which offers a 15% discount to AES members, phone (02) 9997 3973
ISBN: 1-4129-1060-9
This book is an excellent
resource for evaluators who
wish to inform policy decisions
with collective insights
gained from diverse threads
of relevant literature. In the
current international scenario,
systematic reviews with a
priori protocols are attracting
large funding (Suri 2004).
Often, reductionist systematic
reviews oversimplify the role of
evidence in policy formulation
by assuming that policy
decisions should be informed by
aggregated relationships between
individual contextual variables
and outcome variables. This
book is timely in its emphasis
on the complex, multifaceted
nexus between evidence and
policy. Contesting the notion
of social interventions as
‘treatments’ aimed at ‘subjects’,
Pawson emphasises that social
interventions are ‘complex
systems thrust amidst complex
systems’ (p. 168). Recognising
that most policy decisions are
made in politically charged
environments that are constantly
changing, Pawson makes
sensible suggestions on how
to realistically inform policy
decisions with the relevant
evidence by deliberately straying
‘outside the policy domain in
question’ (p. 173).
Salient features of Pawson’s
realist synthesis include:
focusing on individual ■
‘mechanisms, contexts and
outcomes’ to investigate
‘for whom, in what
circumstances, and in
what respects a family of
programmes work’ (p. 2)
learning from successful ■
as well as unsuccessful
implementations of a
program
purposefully drawing ■
from ostensibly disparate
literature to identify
underlying mechanisms and
theories that could impact
upon the implementation
and outcomes of a given
intervention
systematic involvement of key ■
stakeholders throughout the
synthesis
regarding a fragment of ■
evidence or a program theory,
rather than a study, as a unit
of analysis
evaluating any fragment of ■
evidence in the context of
how it contributes towards
constructing a synthetic
understanding rather than
employing a priori hierarchy
of evidence
learning from, rather than ■
controlling for, ‘diversity,
change, idiosyncrasy,
adaptation, cross-
contamination and program
“failure”’ (p. 178)
tentative fi ndings that tell ■
decision-makers of likely
implications of different
decisions in different
situations rather than telling
what works
acknowledging inherent ■
selectivity in any synthesis
process.
Evaluators with varied
levels of expertise in research
synthesis methods can draw on
this book. Experienced research
synthesists can fi nd some useful
strategies for improving their
research syntheses. Novices
would fi nd this book a good
introduction to meta-analytic
systematic reviews and ways in
which realist syntheses move
beyond systematic reviews.
However, in a book that contests
the hegemony of meta-analytic
systematic reviews, I would have
liked more acknowledgement
of the rich literature on diverse
forms of research synthesis
methods. It would have been
valuable to see Pawson’s views
on how his method draws
on, and differs from, various
formally proposed qualitative
research synthesis methods.
Helpful descriptions of these
diverse research synthesis
methods are available in the
published literature (e.g. Dixon-
Woods, Agarwal, Jones, Young
& Sutton 2005; Mays, Pope
& Popay 2005). Even though
Pawson is inclusive of qualitative
and quantitative research in
general, he has shied away from
clarifying the role of postmodern
and emancipatory research
reports. Given that openly
ideological research is gaining
popularity in many circles of
social sciences, this is a serious
omission. At times I found an
excessive use of metaphors and
analogies distracting, and this
also made some sections diffi cult
to read.
Despite these caveats,
Pawson’s realist synthesis offers
useful guidelines to synthesise
evidence for:
examining the integrity of a ■
program theory
adjudicating between rival ■
program theories
investigating the suitability of ■
the same theory in different
settings
comparing the offi cial version ■
of a theory with its actual
implementation in practice.
I would urge prospective
readers to skim through this
book’s last chapter published
To continue reading
Request your trial