Book Review: Evaluation in Action: Interviews with Expert Evaluators
| Author | Scott Bayley |
| DOI | 10.1177/1035719X1101100111 |
| Published date | 01 March 2011 |
| Date | 01 March 2011 |
| Subject Matter | Book Review |
BOOK REVIEWS
Evaluation Journal of Australasia, Vol. 11, No. 1, 2011
52
We really don’t know much
about how people practise
evaluation. Journal articles
and books are written about
evaluation models and how
to conduct an evaluation—
describing purposes, steps to be
taken in planning, methods for
collecting data—and facilitating
the use of results. But we know
very little about what evaluators
actually do in practice. How do
they identify the evaluation’s
purpose? What stakeholders do
they involve in that decision?
How do they make decisions
concerning what to measure?
How do they decide who to
involve in analysing the data?
What strategies do they use to
encourage use?
Evaluation in Action takes
readers behind the scenes of real
evaluations and introduces them
to the issues faced and decisions
made by notable evaluators in
the field. The book builds on
‘Exemplars’, a popular section
in the American Journal of
Evaluation, in which a well-
known evaluator is interviewed
about an evaluation he or she
has conducted. Through a
dialogue between the evaluator
and the interviewer, the reader
learns about the problems the
evaluator faced in conducting the
evaluation and the choices and
compromises he or she chose to
make.
The book’s near 500 pages
are organised into 15 chapters.
The heart of the book is the 12
interviews illustrating a variety of
evaluation practices in different
settings (mostly health, welfare
or education programs), along
with commentary and analysis
concerning what each interview
teaches us about evaluation
practice. The book also includes
an alternative table of contents
and a discussion of the different
ways in which the interviews can
be grouped and read, which will
help students and practitioners
looking for more information
and insight on a particular issue.
Part 1, ‘Traditional
Evaluations: Evaluations with
the Primary Purpose of Judging
Merit and Worth’, contains three
interviews. The interview with
James Riccio provides a well-
written example of a politically
sensitive evaluation of a welfare-
to-work program conducted
by an external evaluator. The
evaluation discusses how
the evaluation was managed
with attention to stakeholder
involvement, focusing on both
program implementation and
impact, and the challenges
of communicating complex
results to people who are not
familiar with research results.
The question being addressed
was not whether or not welfare
reform works, but rather what
aspects of it works, with what
type of recipients, and in what
kind of settings. The interview
with Jennifer Greene concerned
a leadership program; the
evaluation took a responsive
approach with an emphasis on
using qualitative methods to
describe the program and reach
judgements. In contrast, the
interview with Len Bickman
concerned a large and complex
evaluation of mental health
services for children, with
an emphasis on testing the
theory underlying the program.
Bickman makes the point that
the problem for evaluators is
that most programs are not
well designed and this occurs
due to the incentives facing
both policymakers and service
providers and the nature of their
interactions. I really enjoyed
reading this particular interview.
Part 2, ‘Evaluations with a
Focus on Descriptions’, contains
three interviews. The interview
with David Fetterman was my
favourite part of the book. The
evaluation was a situationally
responsive study of a teacher
education program by an internal
evaluator. This was an excellent
illustration of how an expert
evaluation practitioner will adapt
their choice of approach to suit
the nature of the program and
the context of the evaluation
itself. The interview with Debra
Rog concerning her evaluation
of a homeless families program
portrayed a descriptive and
exploratory approach that
evolved during the five years
of the evaluation. Rog also
discusses her approach to using
multiple methods to disseminate
the evaluation’s findings. The
interview with Gary Henry
concerned his development of
a framework for performance
monitoring in a school system. I
thought this chapter was weak
and not really in keeping with
the rest of the book. It was about
monitoring not evaluation, and
the discussion of the issues was
somewhat superficial.
Part 3, ‘Evaluations with an
Emphasis on Program Planning
and Organizational Development’
contains four interviews. Jean
King discuses her evaluation of
an educational program, with
a focus on participation and
evaluation capacity building.
Stuart Donaldson talks about
his comprehensive evaluation of
an employment program from
a theory-driven approach. An
unusual aspect was how the
evaluation team actively sought
Title: Evaluation in Action: Interviews with Expert Evaluators
Authors: Jody Fitzpatrick, Christina Christie and Melvin M. Mark
Publisher/year: Sage, Los Angeles, 2009
Extent/type: 456 pages, paperback
Price: AU$69/NZ$94 from Footprint Books which offers a 15% discount to AES members, phone +61 2 9997 3973,
email <info@footprint.com.au>, website <http://www.footprint.com.au> for the latest prices
ISSN: 978-1-4129-4974-3
EJA_11_1.indb 52 6/11/11 5:37:10 PM
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting