Evaluation: A Booming Business but is it Adding Value?1
| Author | Frans L Leeuw |
| Date | 01 March 2009 |
| DOI | 10.1177/1035719X0900900102 |
| Published date | 01 March 2009 |
| Subject Matter | Refereed Article |
Leeuw—Evaluation: a booming business but is it adding value? 3
Evaluation Journal of Australasia, Vol. 9, No. 1, 2009, pp. 3–9
REFEREED ARTICLE
Evaluation: a booming business
but is it adding value?1
Evaluation, monitoring, inspection and performance
auditing have become a booming business. The
numbers of professional societies have strongly
increased and there are so many studies that authors
recently referred to them as ‘streams’ and ‘fleets’.
Evaluation has also been called a ‘positive social
epidemic’. This article reveals that the booming business
not only has positive consequences and ‘added value’,
but also unwanted downsides. ‘Evaluation machines’,
‘death by evaluation’, ‘evaluitis’ and the ‘performance
paradox’ are examples.
This article presents three possible explanations for
the situation in which a booming business goes hand
in hand with the beginnings of diminishing returns
of evaluation for society. The first explanation is that
evaluations are becoming part of management support
systems that produce information that often only
reinforces management decisions. The second looks
into the unintended and negative consequences of
‘paradigm wars’. And the third suggestion deals with the
limited explanatory power of evaluations.
The problem
Evaluation, monitoring, inspection and performance auditing are no longer ‘baby’
or ‘infant’ industries’. In fact, the number of studies and reports has increased
so much over the last two decades that Rist & Stame (2006) gave their latest
book the title: From Studies to Streams [of studies]. Some years earlier, Weiss
(1998) used the metaphor ‘fleets of studies’. The number of jobs for evaluators
has also increased (Leeuw, Toulemonde & Brouwer 1999; Preskill 2008), as
has the numbers of (national) evaluation societies (Furubo, Rist & Sandahl
2002) and their memberships. Also, since 2001, membership of the American
Evaluation Association, for example, has grown more than 79 per cent (from
3055 members in 2000 to 5479 by December 2007). Therefore, Preskill (2008)
coined the idea of evaluation becoming a (positive) social epidemic. Part of this
‘epidemic’ are the ‘systems of evaluations’ that have their own organisations,
handbooks, methodology, money and staff (Leeuw & Furubo 2008). Examples
are: performance auditing and monitoring, M(onitoring and) E(valuation),
experimental evaluations and evaluation and accreditation.
For those believing that these booming businesses only lead to positive
consequences and ‘added value’, this article brings bad news. There are rather
unpleasant side effects. In his speech for the 2008 Congress of the American
Frans L Leeuw
Frans L Leeuw is Professor of Law,
Public Policy and Social Science
Research at the University of
Maastricht, the Netherlands, and
Director of the Netherlands Justice
Research Center (WODC), Den
Haag, and is also President of the
Dutch Evaluation Society.
Email: <frans.leeuw@
maastrichtuniversity.nl> or
<flleeuw@cuci.nl>
EJA_9_1.indb 3 18/10/09 10:34:17 PM
To continue reading
Request your trial