Developmental evaluation during the COVID-19 pandemic: Practice-based learnings from projects in British Columbia, Canada
| Author | Ihoghosa Iyamu,Mai Berger,Saranee Fernando,M Elizabeth Snow,Amy Salmon |
| DOI | http://doi.org/10.1177/1035719X221119841 |
| Published date | 01 March 2023 |
| Date | 01 March 2023 |
| Subject Matter | Practice Articles |
Practice Article
Evaluation Journal of Australasia
2023, Vol. 23(1) 23–39
© The Author(s) 2022
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1035719X221119841
journals.sagepub.com/home/evj
Developmental evaluation
during the COVID-19
pandemic: Practice-based
learnings from projects in
British Columbia, Canada
Ihoghosa Iyamuand Mai Berger
School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada
Centre for Health Evaluation and Outcome Sciences (CH´
EOS), Vancouver, BC, Canada
Saranee Fernando
Centre for Health Evaluation and Outcome Sciences (CH´
EOS), Vancouver, BC, Canada
M Elizabeth Snowand Amy Salmon
School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada
Centre for Health Evaluation and Outcome Sciences (CH´
EOS), Vancouver, BC, Canada
Abstract
In this article, we explore experiences and learnings from adapting to challenges
encountered in implementing three Developmental Evaluations (DE) in British Co-
lumbia, Canada within the evolving context of the COVID-19 pandemic. We situate
our DE projects within our approach to the DE life cycle and describe challenges
encountered and required adaptations in each phase of the life cycle. Regarding
foundational aspects of DEs, we experienced challenges with relationship building,
assessing and responding to the context, and ensuring continuous learning. These
challenges were related to suboptimal embeddedness of the evaluators withi n the
evaluated projects. We adapted by leveraging online channels to maintain commu-
nications and securing stakeholder engagement by assuming non-traditional DE roles
based on our knowledge of the context to support project goals. Additional challenges
Corresponding author:
Amy Salmon, Centre for Health Evaluation and Outcome Sciences (CH´
EOS), 588-1081 Burrard Street,
Vancouver, BC V6Z1Y6, Canada.
Email: asalmon@cheos.ubc.ca
experienced with mapping the rationale and goals of the projects, identifying domains
for assessment, collecting data, making sense of the data and intervening were adapted
to by facilitating online workshops, collecting data online and through proxy evaluators,
while sharing methodological insights within the evaluation team. During evolving
crises, like the COVID-19 pandemic, evaluators must embrace flexibility, leverage, and
apply their knowledge of the evaluation context, lean on their strengths, purposefully
reflect and share knowledge to optimise their DEs.
Keywords
COVID-19, developmental evaluation, utilisation-focused evaluation
Background
Due to the COVID-19 pandemic, organisations and programs have needed to con-
tinuously adapt to rapidly changing, unpredictable contexts. Likewise, evaluations
have been required to adapt to evolving evaluation contexts and shifting organisational
priorities (Gawaya et al., 2022). Developmental Evaluation (DE), with its focus on
‘inform[ing] adaptive development of change initiatives in complex dynamic envi-
ronments’(Dozois et al., 2010;Patton, 2011,2021), has been suggested as a suitable
evaluation approach to support the adaptation of initiatives and programs during crises
like the COVID-19 pandemic (Karalis, 2020;Patton, 2021). Moreover, ‘adaptation to
crisis’has been recognied as an emergent purpose for DE (Patton, 2021). As the
pandemic persists, practical learnings about applying DE during these extraordinary
times must be shared.
Since March 2020, various perspectives on applying DE during the pandemic have
emerged across various fields, including education, social justice and healthcare
programs. Parker et al. (2021) implemented a DE to understand how to support medical
students and faculty to transform their learning approaches during crisis. They
identified the need for a systems-based, context-driven learning approach that embraces
change, creates space for reflection and relies on real-time information. Using a DE
allowed them to broaden and reframe education scholarship practice during the
pandemic. Other articles describe DE’s suitability for programs implemented during
COVID-19, highlighting the evolving circumstances initiatives must respond to
(Karalis, 2020;Muskopf et al., 2021).
Despite the surge in literature on this topic, not much is known about practical
adaptations developmental evaluators have made to respond to challenges of the
pandemic, including restrictions on in-person gatherings, travel limitations and reduced
access to project sites. Evaluators have also conducted DEs of initiatives while re-
sources and attention were diverted towards pandemic response. While the literature
describes practical steps for implementing DE in COVID-19, most remain theoretical
and unreflective of real-life challenges faced by evaluators (Parker et al., 2021;Patton,
24 Evaluation Journal of Australasia 23(1)
To continue reading
Request your trial