The Deconstruction of Evaluation Research: Part I, The Way Forward?
| Published date | 01 March 1984 |
| Pages | 3-13 |
| DOI | https://doi.org/10.1108/eb055498 |
| Date | 01 March 1984 |
| Author | Dan Gowler,Karen Legge |
| Subject Matter | HR & organizational behaviour |
The reconstruction of Evaluation
Research:
Part I, The Way
Forward?[1]
by Dan Gowler, Templeton College and the University of Oxford, and
Karen Legge, Department of Social and Economic Studies, Imperial College
Introduction
Acts of evaluation—the assessment against implicit or ex-
plicit criteria of the value of individuals, objects, situations
and outcomes—form the core of any high discretion job,
where choices have to be made and decisions taken in a world
of scarce resources. On a day-by-day basis informal evalua-
tions pervade the job of any manager or administrator, but
often this is supplemented by formal evaluation research
studies—whether technology assessment, investment ap-
praisal, the evaluation of markets and competitors, or in the
case of personnel managers—the evaluation of training and
development and of organisational change programmes
generally. These formal studies include the evaluation studies
conducted by "professional" evaluation researchers, such as
those engaged in the evaluation of federally funded US social
change programmes, those drawn from commercial con-
sultancy agencies or occupying an internal consultant's role
within a large company, and those applied social scientists,
working in university departments and research institutions
interested in issues concerning work system and organisa-
tional design
[2].
Many articles published in Personnel Review
attest to this concern with evaluation research [3] and, indeed,
expertise in the conduct of formal evaluation studies has been
identified as a major weapon in the armoury of personnel
managers who adopt a "conformist innovator" approach to
developing their power and influence[4].
In times of recession this concern with evaluation research
is intensified—and not surprisingly so. For, at a deeper
societal
level,
when the climate of opportunity
is
being recast
as one of constraint, evaluation may be seen as a mechanism
(and providing a rhetoric) whereby decision makers can
reconcile the optimistic and pessimistic images of
the
future.
In western industrialised societies we have, on the one hand,
an image of humans as masters of the environment, harness-
ing technological "miracles" to social needs. The image is
captured by the first moon-walkers' carefully considered
epigram "a small step for man, a giant leap for mankind".
On the other hand, though, as economic recessions and
energy crises walk hand in hand, and awareness grows that
high technology brings not only material benefits but poten-
tial social disaster, the image of humans controlling their en-
vironments has come increasingly into question. Slogans
such as "Greenpeace" and "Refuse Cruise" suggest a world
where people are servants or victims of the environments in
which they
live,
not masters or
exploiters.
Evaluation research
gains some of
its appeal by appearing to support the necessity
of
change whilst
admitting the
existence
of
constraints.
The
reconciliation of the two, it suggests, lies in developing in-
formed decision making about the selective allocation of
scarce resources, so that change programmes may be sup-
ported which carry a high probability of cost-effectiveness
(however defined). Evaluation research seeks to contribute
to this activity by providing decision makers, directly or in-
directly, with an objective and reliable information base to
facilitate such decision making.
Given this important symbolic function of evaluation
research—mediating conflicting visions of the future—no
less than its more down-to-earth applications, it is perhaps
surprising that such
research
has been confronted by three
crises which demanded resolution if the activity were to
maintain
its
practical
and
academic
credibility[5].
Less sur-
prising, though, is the speed with which researchers have seiz-
ed on various "solutions" which have rapidly achieved the
status of a new orthodoxy[6]. It is against this background
that this two-part article comes to be written, and its pur-
pose is threefold. First, in Part I
we
intend to outline briefly
the nature of the crises confronting evaluation research and
the conventional strategies which have quickly emerged to
overcome them. Secondly, using as a model the approach
of a relatively new technique of analysis more conventionally
found in literary criticism—"deconstruction"—we intend to
question the validity and effectiveness of these strategies.
Thirdly, in Part II
we
offer our own conclusions as to whether
and how the problems which confront evaluation research
may be overcome.
Crises in Evaluation Research and Conventional Cures
The problems confronting evaluation research which came
to a head in the mid to late 1970s concerned its utilisation,
methods and values. Put briefly, commentators argued that,
in spite of its theoretical values, in practice the findings of
evaluation research were not used, its favoured research
methods (quasi-experimental and correlational survey studies
firmly rooted within a realist ontology and positivistic
epistemology) did not produce "true" findings and that far
from facilitating change, evaluation research supported the
status quo. Furthermore the ethics of many of its practi-
tioners were questionable[7].
While the details of these criticisms may be found
elsewhere[8] suffice it to say that those about utilisation
focused on concerns about the irrelevancy and unrespon-
siveness of findings to those who might be in a position to
use them. Disquiet about methods concerned the practicali-
ty of using experimental and quasi-experimental methods in
field settings, the tension between the requirements of in-
ternal and external validity in such designs, and, at a deeper
level, the legitimacy and appropriateness of
a
realist ontology
PR 13,3 1984 • 3
To continue reading
Request your trial