An examination of potential biases in research designs used to assess the efficacy of sex offender treatment
| Published date | 12 October 2015 |
| Pages | 204-222 |
| DOI | https://doi.org/10.1108/JACPR-01-2015-0154 |
| Date | 12 October 2015 |
| Author | Anthony Beech,Nick Freemantle,Caroline Power,Dawn Fisher |
| Subject Matter | Health & social care,Criminology & forensic psychology,Aggression, conflict & peace |
An examination of potential biases in
research designs used to assess the
efficacy of sex offender treatment
Anthony Beech, Nick Freemantle, Caroline Power and Dawn Fisher
Professor Anthony Beech is a
Professor in Criminological
Psychology at School of
Psychology, University of
Birmingham, Birmingham, UK.
Professor Nick Freemantle is
based at Primary Care and
Population Health, University
College London, London, UK.
Dr Caroline Power is a Forensic
Psychologist at Psychology
Department, Partnerships in
Care, Essex, UK.
Dr Dawn Fisher is the Head of
Psychology at St. Andrews
Healthcare, Birmingham, UK
and The University of
Birmingham, Birmingham, UK.
Abstract
Purpose –The purpose of this paper is to examine the potential biases in research designs used to assess
the efficacy of sex offender treatment.
Design/methodology/approach –In all, 50 treatment studies (n ¼13,886) were examined using a random
effects meta-analysis model.
Findings –Results indicated a positive effect of treatment for both sexual (OR ¼0.58, 95%, CI 0.45-0.74,
po0.0001), and general recidivism (OR ¼0.54, 95%, CI 0.42-0.69, po0.0001), indicating that the
likelihood of being reconvicted after treatment was around half compared to no treatment. RCTs showed no
significant effect for sexual or general, recidivism. Significant effects were found for non-RCT designs
(i.e. incidental cohort, completers vs non-completers designs). Assignment based on need (i.e. giving
treatment to those who were high-risk) indicated a negative effect of treatment.
Practical implications –The results highlight the importance of considering study design when considering
treatment efficacy.
Originality/value –The current research reports studies identified up until 2009, and examined both
published, and unpublished, research originating from a variety of samples employing a random effects
model. Consequently, it can be argued that the results are both original and are reflective not only of identified
studies, but are also representative of a random set of observations drawn from the common population
distribution (Fleiss, 1993). The results of the study suggest that what is required in future research is
methodological rigour, and consistency, in the way in which researchers measure the effectiveness of sexual
offender treatment.
Keywords Meta-analysis, Treatment, Sex offenders, Biases, Random effects models, Research designs
Paper type Research paper
Introduction
Demonstrating treatment effectiveness is probably the most contentious issue in the field at the
present time. Research studies can be interpreted very differently and experts regularly disagree
upon the value of studies given the methodological flaws often present within this type of
research (Hanson, 1997). Therefore, there is little consensus as to whether treatment works (see
Brooks-Gordon et al., 2006; Harkins and Beech, 2007; Marshall and Marshall, 2007, for
discussions of this issue) . There are a number of facto rs that have contributed t o
the lack of accord. The main issue is centred on the confidence in which we can say that the
wide mixture of study, and quality, design have the potential to demonstrate that therapy has
an impact. Some have argued (e.g. Harris et al., 1998; Quinsey et al., 1993; Rice and Harris,
1997, 2003), that unless a randomized control trial (RCT) design is employed (i.e. where
Received 21 January 2015
Revised 27 April 2015
16 June 2015
Accepted 16 June 2015
The authors would like to thanks
Professor Theresa Gannon for her
helpful comments on an earlier
draft of the paper.
PAG E 20 4
j
JOURNAL OF AGGRESSION, CONFLICT AND PEACE RESEARCH
j
VOL. 7 NO. 4 2015, pp.204- 222,© Emerald Group Publishing Limited, ISSN 1759-6599 DOI 10.1108/JACPR-01-2015-0154
participants are assigned by chance to treatment/control, and where pre-existing differences
between the two groups will be randomly distributed), the question of whether treatment works,
in terms of reducing recidivism rates, can never be properly answered.
The perceived scientific rigour of the RCT approach has led to the assertion that this is the gold
standard design for evaluating any type of treatment (see e.g. Egger et al., 2005). This is not a
new assertion, as nearly 80 years ago, Fisher (1935) noted that, “It may be said that the simple
precaution of randomization will suffice to guarantee the validity of the test of significance, by
which the result of the experiment is to be judged.”(p. 21). Despite the perceived scientific rigour
provided by RCTs they are rarely applied within criminal justice settings, or even for that matter
in sex offender treatment studies, even though this is the strongly preferred design by the
Association for the Treatment of Sexual Abusers (ATSA), see www.atsa.com/ppRCTuse.html
and www.atsa.com/ppRCTimplement.html for a thorough exposition of the implementation of
such a design.
RCT designs, however, are not without their own associated problems. For example, for this
design to be effectively implemented, treatment for some individuals will have to be deliberately
withheld to create a “non-intervention”group. This raises ethical and legal issues for major
institutional systems, given the possible consequences of denying a high-risk sexual offender
treatment (Harkins and Beech, 2007; Marshall and Marshall, 2007). In addition, unless a sample
is sufficiently large, random allocation cannot adequately ensure that treatment/control groups
are reasonably balanced (Marques and Murphy, 2004). As regards the latter problem, techniques
such as “minimization”(which aims to reduce the imbalance between the number of individuals in
each treatment group over a number of factors) can be used to help improve the efficacy of the
approach (Bracken, 2001). Hence, more typically, when evaluating sex offender treatment,
efficacy designs have included: incidental cohort designs where control groups may have been
taken from a different time period, or from a sample that could not be offered treatment, or
possibly because they were released prior to the implementation of the treatment programme
under study (e.g. Hanson et al., 1993; Proctor, 1996); comparing volunteers undertaking
treatment with treatment refusers (Barnes and Peterson, 1997); comparing treatment completers
vs treatment drop-outs (Seager et al., 2004); and assignment based on “need”. In these designs
it is clear that there are potential confounds that will influence the outcomes of such studies in
terms of inflating differences between those undertaking treatment compared to controls. Some
argue that non-RCT designs are so inherently biased that they should not be used at all.
As regards to the potential biases in these designs, changes in rates of sexual offending over time
can have effects upon incidental cohort designs, where the recidivism rates of treated offenders
are compared to an untreated cohort of offenders released in previous years (e.g. Hanson et al.,
2004). For example, Mishra and Lalumière (2009), and the Minnesota Department of Corrections,
have noted that recidivism rates have declined since the 1990s. In the latter case this has been
attributed to the State’s increase in the length and intensity of post-release supervision
(Minnesota Department of Corrections, 2007). These findings have clear implications in this
design in that if cohorts are used that are not contemporaneous with the treated individuals, an
artificial difference may be found irrespective of whether treatment has been undertaken.
For volunteer/treatment refusal designs, motivation and subsequent selection bias (whether
candidates really participated voluntarily, or were referred by the associated administration
dealing with them) have been shown to confound treatment outcome (Jones et al., 2006).
In completers/drop-out designs, Pelissier (2007) found that higher levels of motivation and
education, and beginning treatment within three months of initial imprisonment, were associated
with treatment completion. Therefore, the control groups are much more likely to be less well
educated, unmotivated, and hence possibly more risky. Beyko and Wong (2005) found that
non-sexual criminogenic needs (i.e. aggression, rule-violating behaviours), and responsivity
factors (i.e. denial, lack of motivation) significantly differentiated programme non-completers from
completers, particularly with rapists.
Assignment on the basis of need designs, by definition, means that higher risk individuals
are offered treatment in preference to lower risk controls. Here the best that may be expected
in treatment is a reduction in recidivism rates of those high-risk individuals to those of low
VOL. 7 NO. 4 2015
j
JOURNALOF AGGRESSION, CONFLICTAND PEACE RESEARCH
j
PAGE205
To continue reading
Request your trial