Boosting or Hindering Aid Effectiveness? An Assessment of Systems for Measuring Donor Agency Results
| Published date | 01 February 2016 |
| Author | Sarah Holzapfel |
| DOI | http://doi.org/10.1002/pad.1749 |
| Date | 01 February 2016 |
BOOSTING OR HINDERING AID EFFECTIVENESS? AN ASSESSMENT OF
SYSTEMS FOR MEASURING DONOR AGENCY RESULTS
SARAH HOLZAPFEL*
German Development Institute/Deutsches Institut für Entwicklungspolitik (DIE), Germany
SUMMARY
Amid rising criticism of aid effectiveness coupled with tight budgets in many donor countries at a time of economic crisis, donor
agencies are under pressure to deliver value for money and to demonstrate development results. In response to these pressures,
more and more donor agencies are adopting standard indicators, which allow for results to be aggregated across interventions
and countries, in order to report agency-wide results. This article analyses the reporting practices of eleven bilateral and multi-
lateral donor agencies and assesses the implications of agency-wide results measurement systems for aid effectiveness. The anal-
ysis shows that the data on aggregate results provided by donor agencies is only of limited informational value and does not
provide an adequate basis for holding donor agencies to account. Moreover, reporting on agency-wide results may have a
number of adverse effects. Given the various limitations and risks identified in this article, I suggest that donor agencies
should explore complementary options or alternatives to standard indicators in order to meet their reporting requirements.
Donor agencies are advised to invest more in rigorous impact evaluations and to raise the transparency of individual interven-
tions. © 2016 The Authors. Public Administration and Development published by John Wiley & Sons Ltd.
key words—corporate results frameworks; standard indicators; accountability; results reporting; development cooperation; aid
effectiveness; results agenda
INTRODUCTION
The international aid system has pushed for greater accountability and a clearer results orientation during the past two
decades. Despite being seen in the past as “angelic deliverers of urgently needed sustenance”(Ramalingam 2013,
106), aid agencies today are under increasing public scrutiny and must demonstrate evidence of the positive effects of
development cooperation. In response to concerns about the effectiveness of aid and in order to demonstrate ‘value for
money’in times of the economic crisis, a growing number of bilateral and multilateral development agencies have
adopted results frameworks for monitoring and managing progress in pursuing strategic objectives and for reporting
on agency-wide performance. These results frameworks involve the use of a selected set of standard indicators for aggre-
gating intervention results across countries in order to provide the public with an overview of their performance and their
contributions to overarching development goals. The idea is that the higher level of accountability produced by the use of
standard indicators will help to sharpen the focus on results, boost aid effectiveness and bolster public support for devel-
opment cooperation.
While everyone is of course in favour of disseminating results, there is mounting criticism of the ‘results agenda’.In
practice,the latter is often gearedmore towards providing accountability to funders than towardsusing results informa-
tion for management and learning purpose so as to improve aid effectiveness (Natsios 2010, 2–3; Barder 2012). In ad-
dition, many commentators have pointed out that the focus on results may have adverse effects and could conflict with
the commitments made as part of the aid effectiveness agenda to strengthen country ownership, harmonise with other
donors and align aid with partner countries’priorities (Hudson and Jonsson 2009; Sjöstedt 2013).
*Correspondence to: S. Holzapfel, German Development Institute/Deutsches Institut für Entwicklungspolitik (DIE), Tulpenfeld 6, 53113 Bonn,
Germany E-mail: sarah.holzapfel@die-gdi.de
public administration and development
Public Admin. Dev. 36,3–19 (2016)
Published online 4 February 2016 in Wiley Online Library
(wileyonlinelibrary.com) DOI: 10.1002/pad.1749
© 2016 The Authors. Public Administration and Development published by John Wiley & Sons Ltd.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in
any medium, provided the original work is properly cited.
Existing literature on the results agenda in development cooperation mainly focuses on result-based manage-
ment at project level and addresses the challenge of measuring results under complexity as well as tradeoffs
with aid effectiveness principles (for example Cook et al. (1995), Natsios (2010), Eyben (2013), Sjöstedt
(2013) and Ramalingam (2013)). There are only few studies to date that deal with the topic of corporate results
reporting (Vähämäki et al., 2011; Organisation for Economic Co-operation and Development/Development
Assistance Committee (OECD/DAC), 2014; EuropeAID, 2013), and these are limited to presenting examples
of selected agencies’reporting systems and to describing the challenges of aggregate results reporting.
It is important to go beyond simple descriptions and to systematically analyse the results reporting practices of
development agencies for two reasons. First, the pressure to account for agency-wide results shapes development
cooperation practices and is one of the determinants in deciding which results are prioritised by donor agencies.
Second, different corporate results reporting systems also have different positive as well as negative effects. These
effects have to be analysed und understood in order to minimise the risks involved and to ensure that accountability
mechanisms help boost aid effectiveness.
This article aims to address this research gap by comparing the experiences of 11 multilateral and bilateral donor
agencies with reporting results in the form of corporate results frameworks. It goes on to assess the implications of
the findings for aid effectiveness.
For the purpose of this analysis, I drew on the current literature on accountability, public-sector performance
measurement and aid effectiveness, and analysed results frameworks and indicator guidance documents used by
the selected donor agencies. The study also included a series of semi-structured interviews with experts from the
selected agencies, with the aim of gathering additional information on results measurement systems and id entifying
the effects of aggregate results reporting on development cooperation practices.
This article is formatted as follows. The next section sets out the background to the study. It summarises the
results agenda and accountability mechanisms in development cooperation and discusses the challenges encoun-
tered in measuring results. Subsequently, the potential risks and adverse effects of results measurement and
reporting are explained. This is followed by a section that explains the methodology used for the empirical analysis.
The next section presents the results of the empirical analysis and explores the differences among donor agencies’
approaches to measuring agency-wide results. Afterwards, the limitations of and the risks associated with the use of
standard indicators are set out. Finally, a number of conclusions are drawn and policy recommendations made.
BACKGROUND TO THE STUDY
The results agenda
The results agenda in development cooperation can be seen as an attempt to make aid more effective and to make
development actors more accountable by measuring and reporting on development results (Shamash et al., 2013,
12). There are two main driving factors behind the results agenda.
The first is the adoption of the Millennium Development Goals (MDGs) in 2000. For the first time, the international
community set global goals, targets and indicators for reducing the many dimensions of extreme poverty for the period
to 2015 (OECD/DAC, 2005/2008, 7, MfDR n.d.). It was recognised by the donor community that a substantial increase
in aid alone would not be enough to achieve the MDGs. Donor agencies pledged to also make aid more effective which
increased the need to measure and monitor the results of development cooperation activities (MfDR n.d.).
The second driving factor is the efforts made by the members of the Organisation for Economic Co-operation
and Development (OECD) to improve public-sector performance by introducing a system of new public manage-
ment (NPM). NPM was first introduced in the 1980s and is characterised by a shift in emphasis from process ac-
countability (i.e. have inputs been used in accordance with the rules?) towards accountability for results (i.e. has the
money been spent efficiently and effectively?) (Saltmarshe et al., 2003, 23; Eyben, 2013, 13).
1
1
While NPM may be seen as a global phenomenon, there are substantial disparities among countries in terms of the rate of adoption of new
public management (NPM) practices and the impact of the NPM reforms. Anglophone countries (e.g. Australia, New Zealand, the UK and
the USA) are leading the reform agenda (Bach and Bordogna, 2011, 2290; Halligan, 2010, 84) and have also been quickest to apply NPM to
development cooperation agencies.
4S. HOLZAPFEL
© 2016 The Authors. Public Administration and Development published by John Wiley & Sons Ltd. Public Admin. Dev. 36,3–19 (2016)
DOI: 10.1002/pad
To continue reading
Request your trial