Some basic observations on conducting a systematic review: a brief reply to Goldstein, Vatalaro and Yair
| Date | 18 June 2018 |
| Pages | 57-63 |
| Published date | 18 June 2018 |
| DOI | https://doi.org/10.1108/JCS-11-2017-0051 |
| Author | Beng Huat See,Stephen Gorard |
| Subject Matter | Health & social care,Vulnerable groups,Children's services,Sociology,Sociology of the family,Children/youth,Parents,Education,Early childhood education,Home culture,Social/physical development |
Some basic observations on conducting
a systematic review: a brief reply to
Goldstein, Vatalaro and Yair
Beng Huat See and Stephen Gorard
Abstract
Purpose –The purpose of this paper is to provide a response to Goldstein et al.’s (2017) attempted rebuttal
of the authors’prior paper in this journal (See and Gorard 2015).
Design/methodology/approach –The prior paper reported a systematic review of interventions to
involve engaging parents more in their children’s education in order to raise school attainment. Goldstein
et al. make a large number of unwarranted claims about its quality. They reproach the authors for using
reports of unpublished evidence, for mis-labelling or mis-describing studies, and for denigrating studies
by labelling them as “bad”. The authors were very surprised when first alerted to this response and went
back to look at all of the research reports that Goldstein et al.claimed the authors mis-represented in the
authors’assessment.
Findings –The authors found that the Goldstein et al. claims are false and based on such a poor
understanding of how evidence is reviewed that it was strange to see their paper in this journal.
Originality/value –In the authors’reply, they look first at why unpublished material must be included in a review,
and why the outlet for publication is not relevant, then at appropriate designs for causal questions, and at the
confusion in Goldstein et al. between evaluation quality and intervention impact. The authors look at many
examples where the confusion leads to Goldstein et al. making incorrect assertions about the authors’paper, in
order to make the point that their whole idea of how to conduct a systematic review is wrong.
Keywords Education,Systematic review, Parental involvement, Parental engagement, Robust evaluations,
School attainment
Paper type Research paper
This paper is written in response to an attempted rebuttal of our prior paper in this journal (See and
Gorard, 2015), by Goldstein et al. (2017). Our paper reported a systematic review of interventions to
engage parents more in their children’s education, in order to raise school attainment. Goldstein
et al. make a large number of unwarranted claims about the quality of our paper which reported a
systematic review. They reproach us for using reports of unpublished evidence, for mis-labelling or
mis-describing studies, and for denigrating studies by labelling them as “bad”.Wewerevery
surprised when first alerted to this response and went back to look at all of the research reports that
Goldsteinet al. claimed we mis-represented in our assessment. We found that their claims are false
and based on a very poor understanding of how evidence is reviewed. In our reply, we look first at
why unpublished material must be included in a review like ours, and why the outlet for publication is
not relevant, then at appropriate designs for causal questions, and at the confusion in Goldstein et al.
between the quality of an evaluation and the impact of an intervention. We look at many examples
where the confusion leads to Goldstein et al. making incorrect assertions about our paper, in order
to make the point that their whole idea of howtoconductasystematicreviewiswrong.
The development of the review
This brief paper is a response to an attempted rebuttal to a report of a systematic review we
previously published in this journal (See and Gorard, 2015). The paper has substantive
Received 21 November 2017
Revised 20 March 2018
Accepted 24 April 2018
Beng Huat See is Associate
Professor and Stephen Gorard
is Professor of Education, both
at the School of Education,
Durham University,
Durham, UK.
DOI 10.1108/JCS-11-2017-0051 VOL. 13 NO. 2 2018, pp. 57-63, © Emerald Publishing Limited, ISSN 1746-6660
j
JOURNAL OF CHILDREN'S SERVICES
j
PAG E 57
To continue reading
Request your trial