Evaluating Contracted Domestic Violence Programs
| Author | Andrew Day,Donna Chung,Ed Carson |
| DOI | 10.1177/1035719X0900900103 |
| Published date | 01 March 2009 |
| Date | 01 March 2009 |
| Subject Matter | Refereed Article |
R E F E R E E D A R T I C L E
E v a l u a t i o n J o u r n a l o f A u s t r a l a s i a , V o l . 9 , N o . 1 , 2 0 0 9 , p p . 1 0 – 1 9
Evaluating contracted domestic
violence programs
Standardisation and organisational culture1
The growing significance of domestic violence
Ed Carson
programs run by the state and contracted non-
Donna Chung
government agencies in Australia over the past decade
has not been matched by a commensurate growth in
Andrew Day
evaluation of those programs. A number of evaluations
have been conducted, but only a few have been
long-term and large-scale, due to funding and other
constraints. The promotion of evidence-based practice
and policy encourages practitioners and scholars
to aim for comprehensive and systematic review of
initiatives, but achieving that is no mean feat since both
the service delivery and governance of these programs
are often complex and multi-stranded.
In this article we reflect on theoretical and practical
issues of evaluation by reporting on the experience
of undertaking an evaluation of domestic violence
perpetrator and victim support programs delivered
for an Australian state government by contracted
community service agencies. The experience was
not as we had expected on the basis of social policy
debates. In this situation, there were difficulties in
generating sufficiently robust data on offenders in
group programs that delayed and limited the scope
of a quasi-experimental assessment of intervention,
although that strand continues to date. At the same
time, however, this strengthened the importance
Ed Carson (top left) is Professor of Social
of evaluating service delivery and governance
Policy in the School of Psychology, Social
issues within a ‘realist evaluation’ paradigm. Since
Work and Social Policy at the University of
contemporary domestic violence programs often
South Australia, Magill Campus, Adelaide.
typically combine coordinated support for women
Email:
and children and behaviour change intervention with
offenders, evaluation needs multi-stranded approaches
Donna Chung (top right) is an Associate
that may draw on paradigms more commonly thought
Professor in Social Work and Director of the
of as being in tension.
Centre for the Study of Safety and Wellbeing at
the University of Warwick, Coventry, England.
Email:
Introduction
Andrew Day (bottom) is an Associate Professor
Private violence has become a public issue over the past few decades and in
at the Centre for Offender Reintegration at
Australia. Two of the more tangible manifestations of this shift have been the
Deakin in the School of Psychology, Faculty
development of emergency accommodation services for women and children
of Health, Medicine, Nursing and Behavioural
and domestic violence legislation. Recent funding initiatives include the joint
Sciences at Deakin University, Geelong.
Email:
10
E v a l u a t i o n J o u r n a l o f A u s t r a l a s i a , V o l . 9 , N o . 1 , 2 0 0 9
EJA_9_1.indb 10
18/10/09 10:34:18 PM
R E F E R E E D A R T I C L E
Commonwealth, states and territories’ Partnerships
Collaboration 2009) and their critics, as discussed
Against Domestic Violence (PADV) program. This
in Pawson (2006) and, more generally, in Meagher
has been responsible for funding a raft of projects to
(2002).
improve practice, and share knowledge about what
Assessment of the social impact of domestic
is deemed to be good practice. A range of these
violence programs is theoretically and
initiatives included research and evaluation projects.
administratively possible at the macro level by
However, it is debatable whether the evidence in the
documenting correlations between a program
field is robust and, moreover, the way good practice
introduction and changes in crime rates, drug use,
is determined remains somewhat contentious.
recidivism etc., and at the meso-level by reporting
This article reports on the experience of a team of
on consequences of participation for particular
university researchers undertaking an evaluation of
categories of participants. Macro-level evaluation
a small number of domestic violence perpetrator and
emphasising victim safety is also possible using, for
victim support programs delivered by not-for-profit
example, measures of unmet demand for services.
community service agencies contracted to do so by an
The Australian Institute of Health and Welfare,
Australian state government. The aim of the project
for example, publishes data on unmet demand for
was to compare different models of intervention
domestic violence shelters as a subset of reporting
across urban and rural sites, review the methods of
on SAAP programs, but constraints on reporting
intervention that were used to promote the safety
make this a vexed area of assessment (AIHW
of women and children victims, and evaluate the
2008). It is argued by critics that such high level
outcomes of the programs. At this stage of the
or abstracted measures do not accurately reflect
analysis, only preliminary observations are available
effectiveness of domestic violence responses because
on the effectiveness of the programs for either
they overlook some key factors about domestic
perpetrators or victims, but the research has raised a
violence (Pence & Shepard 1999). The majority of
number of more general methodological issues about
domestic violence cases are not reported to police;
evaluation of domestic violence programs.
criminal charges related to domestic violence are
In this article we initially outline some of the
not always pursued successfully to prosecution
key debates about common evaluation paradigms
and many women do not seek emergency
in the human services and then discuss ethical and
accommodation in escaping domestic violence.
practical issues that have a bearing on these differing
Therefore, critics argue that there is a need for local
conceptual and disciplinary perspectives. Given the
level data and perspectives in evaluating the impact
increasing trends towards government departments
of domestic violence responses, as the context or
contracting third-sector agencies to deliver domestic
environment is important (Klevens et al. 2008).
violence services (as part of a broader trend of
Nevertheless, large-scale quantitative methodologies
contracting out welfare services), it is necessary to
are available to assess program take-up and benefits,
discuss how funding and reporting requirements
including cost–benefit analysis, and are used
associated with contractualism might have a bearing
occasionally in domestic violence research.
on program design, data collection and assessment
In addition, meso-level evaluation approaches
of programs. This requires consideration of whether
are possible by reporting the impact of a perpetrator
tighter reporting regimes associated with current
behaviour change program on a pool of offenders,
funding of domestic violence programs generate
by measuring changes in the flow of conviction/
particular forms of output and outcome data
referral figures over the duration of the program,
that are more appropriate for some evaluation
as has been undertaken by researchers such as
approaches than others.
Edward Gondolf in North America (Gondolf
These findings and trends are then considered
2002) and others, for example Hendricks et al.
in the light of the authors’ current evaluation of
(2006). Such approaches are complex to administer
domestic violence perpetrator programs that are
even where the participation rates are high and
contracted to third-sector agencies by the state. The
outcomes are easy to identify (notwithstanding
article contrasts the literature with our experiences
debates in the specialist literature about definitions
of evaluating the programs, in order to suggest some
and measurement of outcomes). In domestic
implications for the current, and future, domestic
violence contexts these preconditions for robust
violence program evaluation.
measurement often do not exist.
In a similar vein, PADV commissioned an
economic costing of domestic violence (Access
Evaluation and evidence
Economics 2004) and this can be useful in further
Macro and meta-evaluation
judging whether a certain level of expenditure is
Evidence-based policy and practice are the
producing relatively more changes on a pool of
touchstones of recent debates about social welfare
offenders than expenditure at another level. Even
provision, notwithstanding heated debates about
though sizeable budgets have been allocated to them
what constitutes evidence, how one prioritises
in countries such as Australia, such flow analysis is
different types of data and what methodological
difficult to apply to violence prevention programs
paradigms are privileged in these rankings. This is
that have relatively few numbers of participants.
evident in the dialogue between promoters of the
Shifting focus to the recording and classification
Cochrane and Campbell Collaborations (Campbell
of outcomes of programs is even more difficult,
C a r s o n , C h u n g a n d D a y — E v a l u a t i n g c o n t r a c t e d d o m e s t i c v i o l e n c e p r o g r a m s
11
EJA_9_1.indb 11
18/10/09 10:34:18 PM
R E F E R E E D A R T I C L E
especially where programs are particularly
one cannot be confident that differences observed
concerned with victim safety and...
To continue reading
Request your trial