Assessing achievements in place-based initiatives – developing a tailor-made tool
| Author | Patricia M O’Connor,Eleanor Clune,Tracey Marriner,Shantanu Sheshgir,Jill Waddell |
| Published date | 01 September 2022 |
| Date | 01 September 2022 |
| DOI | http://doi.org/10.1177/1035719X221102278 |
| Subject Matter | Practice Article |
Practice Article
Evaluation Journal of Australasia
2022, Vol. 22(3) 173–192
© The Author(s) 2022
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1035719X221102278
journals.sagepub.com/home/evj
Assessing achievements in
place-based initiatives –
developing a tailor-made tool
Patricia M O’Connor, Eleanor Clune, Tracey Marriner,
Shantanu Sheshgir and Jill Waddell
Australian Healthcare Associates, Australia;
Adjunct Fellow, School of Social Sciences, Western Sydney University, Sydney, Australia
Abstract
Assessing achievements of place-based initiatives (PBIs) has become increasingly im-
portant in contemporary evaluation practice. While key characteristics of successful
PBIs have been described in the literature, the practicalities of assessing implementation
progress across multiple PBIs under the umbrella of a single program invoke complex
challenges. This article presents the process used to develop a tool for assessing
implementation progress across a 14-site PBI aimed at improving health and education
outcomes for First Nations children in Australia. Sites included a mix of metropolitan,
regional, remote and very remote locations around Australia and varying levels of PBI
maturity. A literature scan identified the attributes of successful PBIs and indicators/
measures that could be used to evaluate them. These attributes were condensed into
eight domains, and a further two were added due to their importance for this eval-
uation. The identified indicators and measures were mapped against these domains to
generate a question bank. A process of consistently rating sites against the measures
relevant to each of them enabled the identification of achievements within each site,
comparisons between sites and evaluation of the progress of the program overall. The
development of this tool provides a case study for developing a practical, fit-for-
purpose tool to evaluate PBIs.
Corresponding author:
Patricia M O’Connor, Evaluation and Advisory Division, Australian Healthcare Associates, Level 6, 140
Bourke St, Melbourne, VIC 3000, Australia.
Email: trish.o’connor@ahaconsulting.com.au
Keywords
Place-based initiatives, collective impact, integration, complex evaluation, place-based
initiative assessment tool, success indicators, First Nations Australians
Introduction
Place-based initiatives (PBIs) are designed and delivered with the purpose of targeting
one or more specific geographical location(s) and population group(s) in order to effect
change and respond to ‘wicked problems’–complex, unpredictable or intractable
social or cultural problems that are difficult or impossible to solve (Head & Alford,
2015;Kolko, 2012)–as well as entrenched disadvantage (Moore et al., 2014). The
complex and adaptive nature of PBIs, the diversity of settings in which they are
implemented and the range of issues they are intended to address mean that PBIs are
inherently difficult to evaluate (Heery et al., 2018). Further, as PBIs are, by definition,
specific to a location and community, the evaluation of more than one PBI under the
umbrella of a single program or funding stream poses methodological challenges, as no
‘one size fits all’solution exists (Crew, 2020;Dart, 2018).
This article describes the authors’approach to developing and using a project-
specific evaluation tool to assess the implementation achievements of a program in-
volving 14 PBIs. While the tool itself is tailored to a specific evaluation, we present our
approach to its development to illustrate the key considerations that framed our thinking
and, by doing so, hope to provide evaluators with a template for evaluating the im-
plementation of other complex PBIs.
Background
The evaluand was a national program jointly funded by the Australian Government
Departments of Health and (then) Education and Training (Education). It aimed to
improve health and education outcomes for First Nations children through better
integration of early childhood education, maternal and child health and family support
services. The program was progressively rolled out from July 2016. The evaluation
began in 2017 and, by its conclusion in 2019, a total of 14 sites and 25 funded or-
ganisations were involved in the program. Program funding continued beyond 2019;
thus, allowing sites involved in this evaluation the time needed to develop and embed
the program beyond the evaluation timeframes.
Sites differed considerably in their geographic and demographic profile, duration
and source(s) of funding and their approach to program design and implementation.
Geographically, a mix of metropolitan, regional, remote and very remote locations were
included. Demographically, the proportion of First Nations people in each community
ranged from 2 to 94%. PBI maturity varied between sites; at the close of the evaluation,
some had been in existence for more than 2 years; others only several months. Of the 14
sites, 11 were jointly funded by both departments under separate grant procurement
174 Evaluation Journal of Australasia 22(3)
To continue reading
Request your trial