Practical application of cost-utility analysis in summative evaluation
| Author | Mardi Trompf,Fiona Kotvojs |
| DOI | 10.1177/1035719X20986251 |
| Published date | 01 March 2021 |
| Date | 01 March 2021 |
| Subject Matter | Academic Articles |
https://doi.org/10.1177/1035719X20986251
Evaluation Journal of Australasia
2021, Vol. 21(1) 24 –39
© The Author(s) 2021
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1035719X20986251
journals.sagepub.com/home/evj
Practical application of
cost-utility analysis in
summative evaluation
Mardi Trompf
The University of Melbourne, Australia
Fiona Kotvojs
Kurrajong Hill Pty Ltd, Australia
Abstract
Donors prioritise evaluation of Value for Money (VfM) in development interventions;
however, the theory and practice of doing so is still developing and applied
inconsistently. Theory found in donor government guides and textbooks is often
high-level and economic evaluation theory can be difficult to apply in practice.
This is compounded when there are multiple stakeholder groups, patchy data
quality and short time horizons for decision making. This article demonstrates
how Cost-Utility Analysis (CUA) can be used as a programme evaluation tool
to bring practice together with theory to meet donor needs, suit development
environments and provide evaluation robustness in defensible VfM conclusions.
The example described here is in the evaluation of a programme in Samoa, where
almost AU$10 million was donated by the Governments of New Zealand and
Australia for tourism industry assistance in recovery from the 2012 Tropical
Cyclone Evan (TCE). The programme design had six delivery modalities and its
subsequent evaluation included an analysis of the cost utility of each modality,
feeding into a VfM conclusion. This practical application of CUA theory
demonstrates an effective approach to evaluating VfM.
Keywords
cost-utility analysis, development evaluation, economic indicator, Samoa, value for
money
Corresponding authors:
Mardi Trompf, MarVAL Consulting, Collingwood, VIC, 3066, Australia; The University of Melbourne,
Melbourne, VIC 3010, Australia.
Email: marditrompf_consulting@yahoo.com.au
Fiona Kotvojs, Kurrajong Hill Pty Ltd, Narooma, NSW 2546, Australia.
Email: fiona@kurrajonghill.com.au
986251EVJ0010.1177/1035719X20986251Evaluation Journal of AustralasiaTrompf and Kotvojs
research-article2021
Academic Article
Trompf and Kotvojs 25
Introduction
Donors prioritise Value for Money (VfM) in evaluation of development interventions
(Jackson, 2012); seeking to maintain public accountability, test effectiveness of aid and
determine whether interventions are meaningful. Despite this priority, the authors found
limited practical examples upon which to model an evaluation approach in the field.
Approaches that were identified varied from applying financial ratios through to qualita-
tive discussions and procurement analyses. This article discusses how a new VfM model
could be applied to evaluations. This article draws on learning to date and existing eco-
nomic analysis theory and briefly describes development of VfM theory and practice.
There is little clarity or agreement on what donors require from a VfM analysis
despite this analysis being a priority. Similarly, researchers do not have a consistent
position on what such an analysis would encompass. Tarride et al. (2009, p. 6) describe
VfM analysis as being derived from the use of an economic analysis involving ‘the
comparison of alternative courses of action in terms of both their costs and their con-
sequences.’ They proposed use of economic analyses including cost-effectiveness
analysis (CEA), cost-utility analysis (CUA), return on investment (ROI) and cost fea-
sibility analysis (CFA).
Davidson (2012) subsequently added more substance to the VfM definition and
equates value with Scriven’s (1991) evaluand merit, describing the need for compari-
sons to develop a relative merit judgement. Davidson proposes the use of Scriven’s
(1991) three dimensions of cost (type, source and timing) to develop a ‘comparative
cost-effectiveness checkpoint’ (Davidson, 2012, p. 59). The practical combination of
cost elements and development outcomes into a value judgement is not specified.
Donors describe VfM as a combination of factors. In an Organisation for Economic
Co-operation and Development (OECD)1 guide, Jackson (2012) describes VfM as nei-
ther ‘synonymous with economy’ nor with efficiency; ‘Value for money is about find-
ing the right balance between economy, efficiency and effectiveness and can’t be
assessed through only one of these dimensions in isolation’ (Jackson, 2012, p. 1). She
describes VfM as a three-dimensional framework which is ‘not a tool or a method, but
a way of thinking. . .’ (Jackson, 2012, p. 1). This is consistent with the Department of
Foreign Affairs and Trade (DFAT)2 four-part framework of Economy, Ethics,
Effectiveness and Efficiency (DFAT, 2018).
Other donors use similar expressions of VfM with only nuanced differences. The
New Zealand Ministry of Foreign Affairs (MFAT, 2011) defines VfM as ‘achieving the
best possible development outcomes over the life of an activity relative to the total cost
of managing and resourcing that activity and ensuring that resources are used effec-
tively, economically, and without waste’ (p. 1). United Kingdom’s Department for
International Development (DFID, 2011) describes VfM as ‘about maximising the
impact of each pound spent to improve poor people’s lives’ (p. 2); identifying that
VfM is not simply about choosing the cheapest way.
Cost-benefit-analysis (CBA) and cost-effectiveness-analysis (CEA) are proposed
as analyses suitable for VfM judgements by some donors such as the World Bank,
(Independent Evaluation Group, 2010). DFID also introduces use of an economic
To continue reading
Request your trial