Generalising from qualitative evaluation
| DOI | 10.1177/1035719X21993938 |
| Date | 01 March 2021 |
| Published date | 01 March 2021 |
| Subject Matter | Academic Articles |
https://doi.org/10.1177/1035719X21993938
Evaluation Journal of Australasia
2021, Vol. 21(1) 7 –23
© The Author(s) 2021
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/1035719X21993938
journals.sagepub.com/home/evj
Generalising from
qualitative evaluation
John Guenther
Batchelor Institute of Indigenous Tertiary Education, Australia
Ian H Falk
Universitas Mahasaraswati Denpasar, Indonesia; Universitas Andi Djemma, Indonesia
Abstract
Evaluations are often focused on assessing merit, value, outcome or some other
feature of a programme, project, policy or some other object. Evaluation research
is then more concerned with the particular rather than the general – even more so,
when qualitative methods are used. But does this mean that evaluations should not
be used to generalise? If it is possible to generalise from evaluations, under what
circumstances can this be legitimately achieved? The authors of this article have
previously argued for generalising from qualitative research (GQR), and in this article,
they extrapolate the discussion to the field of evaluation. First, the article begins with
a discussion of the definitions of generalisability in research, recapping briefly on our
arguments for GQR. Second, the differentiation between research and evaluation
is explored with consideration of what literature there is to justify generalisation
from qualitative evaluation (GQE). Third, a typology derived from the literature is
developed, to sort 54 evaluation projects. Fourth, material from a suite of evaluation
projects is drawn from to demonstrate how the typology of generalisation applies in
the context of evaluations conducted in several fields of study. Finally, we suggest a
model for GQE.
Keywords
evaluation outcomes, generalising from qualitative evaluation (GQE), generalising
from qualitative research (GQR), evaluation, model for GQE, qualitative evaluation
Corresponding author:
John Guenther, Batchelor Institute of Indigenous Tertiary Education, P.O. Box 2363, Parap, NT 0804
Darwin, NT, Australia.
Email: john.guenther@batchelor.edu.au
993938EVJ0010.1177/1035719X21993938Evaluation Journal of AustralasiaGuenther and Falk
research-article2021
Academic Article
8 Evaluation Journal of Australasia 21(1)
Introduction
Generalising from qualitative research (GQR) has been an abiding interest of the
authors throughout their careers (Falk & Guenther, 2006; Guenther & Falk, 2019a,
2019b). Of all the projects, 54 have been evaluations in a variety of contexts across
Australia and Indonesia (Arnott et al., 2012; Falk et al., 2006; Guenther et al., 2009).
Commissioners invariably want answers to questions, but a sub-text of their enquiries
is whether these results can be applied in other places, situations or times for the pur-
poses of upscaling or replicating programmes (see Mertens & Wilson, 2018, Chapter
8). The issues we have researched have spanned disciplines and topics, including
social capital, community development, literacy education, education, biosecurity,
domestic violence and child protection. In most cases, the concerns are about improv-
ing outcomes from programmes and assessing impact on programme participants.
This article brings together work from two main fields of interest: first, GQR; and
second, exploring the purposes of evaluation, focusing on how generalising from qual-
itative evaluation (GQE) may be achieved, thereby furthering debate around GQE.
Supporting data from a suite of policy-based evaluation projects is presented, which
illustrate where generalisations from qualitative evaluations have been made. In con-
junction with the literature, this provokes a discussion of a potential model for GQE
and principles which emerge as relevant when considering GQE.
Literature review
Is there a need to differentiate ‘research’ from ‘evaluation’?
It could be argued that there is little need to differentiate between research and evalu-
ation because of the similarities between their associated practices, in terms of data
collection methods, ethical conduct and analysis (Chen, 2018). Russ-Eft and Preskill
(2009) suggest generalisability is not a major goal or concern for evaluation where it
is for research, though this assertion does not mean that generalisability is not possible
from evaluation. Patton (2015) argues that evaluation and research have different pur-
poses: the former is about making judgements about programmes and policies, while
the latter aims to ‘test theory and contribute to knowledge’ (p. 17). Nor does this asser-
tion mean that new knowledge – for example, in the form of a new theory of change
– cannot be used for generalisation. Mathison (2008) suggests that similarities exist:
‘Because evaluation requires the investigation of what is, doing evaluation requires
doing research’ (p. 188). This leaves open the possibility that doing evaluation can
create new generalisable knowledge that, for the purposes of this article, an under-
standing of GQR can be applied to GQE.
Definitions of and arguments for GQR
Vogt (2005) defines generalisability as ‘The extent to which you can come to conclu-
sions about one thing (often, a *population) based on information about another (often,
a sample)’ (p. 131). This definition disguises a contested understanding among research
To continue reading
Request your trial