Coping with the Effectiveness Dilemma: Strategies Adopted by State Auditors
| DOI | 10.1177/0020852399654006 |
| Date | 01 December 1999 |
| Author | Robert Schwartz |
| Published date | 01 December 1999 |
| Subject Matter | Articles |
Coping with the effectiveness dilemma: strategies
adopted by state auditors
Robert Schwartz
The politics of evaluation literature strongly suggests that public sector managers
will not willingly assess the execution and outcomes of their programmes. The
politics of evaluation theory has been set down most notably in the 1970s and
1980s in Aaron Wildavsky’s classic ‘The Self-evaluating Organization’
(Wildavsky, 1972), in numerous publications by Carol H. Weiss (Weiss, 1970,
1973, 1988) and in Dennis Palumbo’s book, The Politics of Program Evaluation
(Palumbo, 1987). Briefly, the theory states: Administrators’ interests in organiza-
tional stability, budget maximization and the promotion of a favourable image
contribute to a general desire to refrain from conducting evaluations which might
show agency programmes in a bad light.
Perhaps as a response to feelings that government evaluations are subject to
political bias, state auditors in many countries have been given a mandate to con-
duct effectiveness audits.1Their independence, objectivity and professionalism
make state auditors attractive alternatives to politically biased agency evalua-
tions.
The high degree of independence enjoyed by state auditors makes them
promising candidates for bearing the responsibility of evaluating public agencies.
State audit laws grant Supreme Audit Institutions (sais) considerable independ-
ence from both the executive and the legislature regarding appointment and dis-
missal of the Auditor General, budgets, personnel management, the choice of
audit topics and the publication of findings (Schwartz, 1991). This independence
is meant to insulate sais from political influences such as those which limit the
efficacy of evaluations conducted by or on behalf of public agencies.
Yet, some commentators note that state auditors are not immune to political
considerations. The paucity of audit resources relative to the task at hand means
that sais are faced with tough decisions in choosing among potential audit jobs —
regarding both which government activities to audit and what type of audit to
conduct (fiscal, managerial or programme). Previous work by Ira Sharkansky
(1984) suggests that the allocation of scarce audit resources is affected by politi-
Robert Schwartz is in the Political Science Department, University of Haifa, Mount
Carmel, Haifa 31905, Israel. CDU 65.0192.3
International Review of Administrative Sciences [0020–8523(199912)65:4]
Copyright © 1999 IIAS. SAGE Publications (London, Thousand Oaks, CA and New
Delhi), Vol.65 (1999), 511–526; 010609
03_IRAS65/4articles 11/11/99 11:05 am Page 511
cal considerations of sais which seek to attract attention to their reports. He notes
that, in choosing audit topics, state audit institutions are invariably influenced by
the desire to be relevant and to attract press and public attention. As one of the
political actors, state auditors vie with other actors for the limelight. In listing the
possible consequences of the expanded role of state audit in Canada, Sharon
Sutherland (1986) notes, along similar lines, that because their power comes from
the media, sais will tend to produce reports that are ‘meaningful’ in lay terms —
meaning audits of recognizable, purposeful programmes.
A number of commentators note that the conduct of effectiveness evaluation
endangers the political neutrality of auditors and can place them at the heart of
political controversy (Sutherland, 1986; Gray et al., 1992; Roberts and Pollitt,
1994; Barzelay, 1997; Broadbent and Laughlin, 1997; Schwartz, forthcoming).2
These dangers stem from (1) difficulties in distinguishing between a determina-
tion that a programme is ineffective and the questioning of the merits of policy
objectives; (2) the room for subjectivity at each stage of the measurement of
effectiveness. Gray et al. (1992: 62), for example, note that effectiveness auditing
takes auditors into ‘politically sensitive’, ‘unfamiliar’, ‘uncertain’ and ‘risky’
territory. Michael Barzelay (1997: 247) claims that ‘because the results of pro-
gram effectiveness audits tend to be politically sensitive, the audit body risks
being drawn into politics in a way that threatens organizational autonomy’.
Some sais — notably Canada’s Auditor General — are legally prevented from
conducting effectiveness audit in order to keep them out of policy questions given
to partisan political controversy.3Canada’s decision was based on the feeling
that effectiveness is often a matter of individual judgement rather than objective
measurement and ‘evaluating program effectiveness, the administration main-
tained, was the role of the parliamentary opposition, not an appointed official;
and if the Auditor General were to become involved in it, he would find himself
in the maelstrom of political controversy’ (Sinclair, 1979). Other auditors, such
as Britain’s National Audit Office (NAO), are mandated to audit effectiveness,
but take great care not to appear to be criticizing the merits of policy objectives
(Dewar, 1991).
Some auditors who have ventured down the slippery slope of effectiveness
evaluation have taken falls.
Gray et al. (1992: 56) report that NAO audits which criticized the effectiveness
of government policy led, in 1991, to a series of attacks on Permanent Secretaries
about the way the NAO was entering the policy arena and becoming political. As
an example, they bring the 1988 NAO report on the government’s sale of the
Rover Group to British Aerospace for £50–100 million below market value.
Israel’s State Comptroller, Justice Miriam Ben Porat, found herself in the
centre of a heated debate with cabinet members and with the top brass of Israel’s
Defence Forces. Following the Gulf War, Justice Ben Porat issued a scathing
report, whose bottom line was that the gas masks supplied to the civilian popula-
tion were an ineffective protection against chemical warfare. Much of the blame
was put on government budget decisions. The Minister of Defence attacked audit
512 International Review of Administrative Sciences 65(4)
03_IRAS65/4articles 11/11/99 11:05 am Page 512
To continue reading
Request your trial