Generating evaluation capital: Lessons learned in meta-evaluations in International Women’s Development Agency
| Author | Tracy McDiarmid,Bethia Burgess |
| Published date | 01 June 2018 |
| Date | 01 June 2018 |
| DOI | http://doi.org/10.1177/1035719X18760871 |
| Subject Matter | Articles |
research-article2018
Article
Evaluation Journal of Australasia
2018, Vol. 18(2) 109 –121
Generating evaluation
© The Author(s) 2018
Reprints and permissions:
capital: Lessons learned
sagepub.co.uk/journalsPermissions.nav
https://doi.org/10.1177/1035719X18760871
DOI: 10.1177/1035719X18760871
journals.sagepub.com/home/evj
in meta-evaluations in
International Women’s
Development Agency
Bethia Burgess and Dr Tracy McDiarmid
International Women’s Development Agency, Melbourne, Australia
Abstract
The strategic allocation of evaluation resources in a resource-constrained environment
is a common challenge shared by governments, non-government organisations, and
the private sector. In 2015-2016 International Women’s Development Agency
(IWDA) embarked on an organisational journey to generate additional evaluation
“capital” by producing meta-evaluations. This article explores how the processes of
evaluation and meta-evaluation contribute to organizational governance in terms of
establishing an evidence-based policy environment, an informed learning platform,
and programming informed by evaluation “capital.”
Keywords
feminist evaluation, meta evaluation, women’s economic empowerment (WEE),
women’s safety and security
The strategic allocation of evaluation resources in a resource-constrained environ-
ment is a common challenge shared by governments, non-governmental organiza-
tions, and the private sector. How do we build evaluation as a durable asset for sound
governance within our organisations? How do evaluations go beyond the project to
inform broad programming and policy objectives? In 2015–2016, International
Women’s Development Agency (IWDA) embarked on an organisational journey to
Corresponding author:
Tracy McDiarmid, International Women’s Development Agency, PO Box 64, Flinders Lane, Melbourne,
Victoria, 8009, Australia.
Email: tmcdiarmid@iwda.org.au
110
Evaluation Journal of Australasia 18(2)
generate additional evaluation “capital” by producing meta-evaluations. Resourcing
a process of drawing together the findings and recommendations of multiple evalua-
tions in a shared thematic area allowed the impact of programming to be assessed
more broadly in terms of holistic contribution to our vision of gender equality for all,
to reveal similarities and differences across our programming contexts and partners,
and to draw out recommendations for our organisation in terms of sustainable impact,
quality implementation, and targeted capacity-building.
This article explores how the processes of evaluation and meta-evaluation contrib-
ute to organisational governance in terms of establishing an evidence-based policy
environment, an informed learning platform, and programming informed by evalua-
tion “capital.” The meta-evaluation process in IWDA will be explored using two
recent examples—first, a synthesis of findings from four evaluations of our Women’s
Economic Empowerment Programming, and second, a synthesis of findings from
three evaluations of our safety and security programming. In this article, we will ini-
tially outline the organisational context in which these evaluations and meta-evalua-
tions were conducted, namely, using the principles of feminist monitoring, evaluation,
and learning, before reflecting on our objectives in using this meta-evaluation
approach. A brief summary of how IWDA applied this approach in our two example
thematic areas will contextualise the key learnings at a project, program, and organi-
sational level and set the scene for how our organisation is applying these learnings.
IWDA’s objectives—feminist principles of monitoring,
evaluation, and learning
As an organisation founded to progress the rights of women, IWDA is committed to
ways of working that embody our vision and purpose. In conducting monitoring, evalu-
ation, and learning, we therefore apply the IWDA values of being feminist, accountable,
collaborative, and transformative. As such, our principles outline that our monitoring,
evaluation, and learning processes do no harm and prioritise safety and security over
information needs; will utilise mixed methods that acknowledge different ways of know-
ing and hierarchies of power; build on and generate knowledge as a resource of and for
the women who create, own, and share it; and drive continuous improvement for real,
lasting change. Adopting a meta-evaluation approach is one step toward ensuring we
apply these principles in our evaluative practices and use evaluations to promote trans-
formative change for the women and communities we serve, for the implementing part-
ners we support, and for the organisation we are and hope to become.
Given the additional time and resources required for a meta-evaluation approach,
IWDA has reflected on our reasons for applying this approach, asking ‘What were we
hoping to achieve with meta-evaluations that may have been unlikely with standalone
evaluations?’ The perceived benefits of a meta-evaluation approach may be conceptu-
alised using the project cycle, namely, problem analysis, design, implementation,
monitoring and evaluation, and learning/re-design. For example, an aggregation of
outcomes, learnings, and recommendations across multiple programming contexts
potentially allows an organisation to see how problems are culturally and contextually
Burgess and McDiarmid
111
specific, but also whether proposed solutions and program theories of change are
appropriately grounded in practical terms. It was also hoped that the relevance of
design differences across projects would be unpacked: Were some project designs
more effective? If so, why? To what extent was impact and effectiveness a result of
individual project implementation?
IWDA was interested in being able to identify and understand common imple-
mentation barriers and success factors so that we could better support quality imple-
mentation with an economy of scale benefit in terms of policy guidance, cross-team
collaboration, and shared capacity-building:
Meta-evaluations and synthesis reports are invaluable in taking findings and trends from a
project to a whole of program and organisation level. They promote intra and inter
organisation learning—including sharing results from different contexts, but within the same
thematic areas, between partners. They allow us to capture information in ways that are
accessible to more diverse audiences. Greater accessibility increases accountability.
Systematic synthesis of achievements and challenges allows us to draw in best practice
standards and analytical frameworks to better conceptualise and plan future work. (Donna
McSkimming, Director of Programs, IWDA)
From past evaluations IWDA was also aware that projects implemented across
vastly different contexts often shared common monitoring, evaluation, and learning
challenges, and it was often difficult for standalone evaluations to trigger significant
changes to policies, processes, or resources.
IWDA’s application of the meta-evaluation approach
To meet accountability and learning objectives, IWDA conducts a series of evaluative
activities every year; however, in 2015–2016 and 2016–2017 IWDA identified a spe-
cific focus on women’s economic empowerment programming and women’s safety
and security programming, respectively.
Women’s economic empowerment project-level evaluations
Under IWDA’s Strategic Plan 2016–2021 (2015), IWDA commits to action across five
main areas in achieving its vision of gender equality for all. IWDA’s work within the
thematic area of women’s economic empowerment seeks the realisation of diverse
women having equal power and opportunity to make and act on economic decisions
and share in economic benefit. Historically, a key component of work in Women’s
Economic Empowerment has included support of women’s savings clubs. During the
2015–2016 financial year, IWDA commissioned project-level evaluations of four
long-term women’s economic empowerment programs based in Timor-Leste and the
Solomon Islands, detailed in Table 1, Appendix 1. All evaluations were aimed at iden-
tifying best practice and lessons...
To continue reading
Request your trial