Identifying the comparative academic performance of secondary schools
| DOI | https://doi.org/10.1108/09578231111146498 |
| Published date | 05 July 2011 |
| Pages | 433-449 |
| Date | 05 July 2011 |
| Author | Linda Bendikson,John Hattie,Viviane Robinson |
| Subject Matter | Education |
Identifying the comparative
academic performance of
secondary schools
Linda Bendikson, John Hattie and Viviane Robinson
Department of Teaching, Learning, and Development, University of Auckland,
Auckland, New Zealand
Abstract
Purpose – One of the features of the New Zealand secondary schools system is that achievement closely
reflects the taught curriculum. The National Certificate of Educational Achievement (NCEA) directly
assesses student achievement on the secondary school curriculum through a combination of
criterion-based internal and external assessments. The nature of NCEA means school-level results not
only reflectstudent achievementbut also the abilityof leaders to organise, deliver,and monitor a relevant
curriculumfor students. This paperaims to describe how NCEAdata were used to developa simple but
fair system to assess the relative performance of secondary schools.
Design/methodology/approach – No standardised measures of performance prior to Year 11 are
available in New Zealand. Nor are student-level data available. In the absence of these, multiple
indicators of gross performance, added value and improvement over time were analysed using a
schools-of-similar-type methodology.
Findings – Results indicated that schools in the low and middle SES communities were more likely
to be improving than others, but these improving schools were also more likely to be already high- or
mid-performing. Low-performing schools were least likely to be improving.
Originality/value – Some advantages of this methodology are its ability to be utilised with any
publicly available standards-based achievement data, its validity as an indicator of leadership and
organisational performance, and its ability to track school performance trends over time.
Keywords Methodology,Schools-of-similar-type, Schoolperformance, School improvement,
Secondary schools,Performance appraisal, Assessment, New Zealand
Paper type Technical paper
Introduction
One of the features of the New Zealand secondary schools system is that achievement
closely reflects the taught curriculum rather than being specified by summative tests
related to a set of specifications (usually translated into standardised tests). On the other
hand, neither student-level data nor school level data prior to Year 11 are available in
New Zealand in any systematic manner. The challenges involved in assessing and
categorising levels of school performance under these circumstances are substantial.
The first challenge is the decision about which student outcomes to measure.
Ideally, since communities expect their schools to pursue a range of valued outcomes,
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0957-8234.htm
Recognition and thanks go to Lorelei King-Salisbury, a Ministry of Education employee, for the
production of the data set from publicly available data that forms the basis of the analysis in this
paper.
Identifying
comparative
performance
433
Received May 2010
Revised August 2010
Accepted October 2010
Journal of Educational
Administration
Vol. 49 No. 4, 2011
pp. 433-449
qEmerald Group Publishing Limited
0957-8234
DOI 10.1108/09578231111146498
multiple indicators of student performance should be used. Among the more critical
and most widely agreed, are achievement performance.
The second challenge concerns the decision to judge school performance in terms of
absolute levels of student performance or in terms of the value added (relating to the
progress students have made during a given period of schooling). The absolute
measure can be defended in terms of the need for all students to achieve to a given level
in order to access further education and the job market. This measure may tell us little,
however, about the contribution of the school to the achievement of its students, hence
the argument that some type of value-added analysis is required in addition to a
measure of achievement performance.
In many contexts, data sets that allow calculation of value-added scores are not
available. In these circumstances a school’s contribution to student achievement can be
estimated by comparing it with schools that serve students from similar backgrounds.
To some extent, such comparisons control for the community and student background
factors that partially determine student achievement, and thus the relative school
contribution can be estimated.
The third challenge is how to choose and interpret multiple indicators of student
achievement for a given school in terms of a category of overall school performance.
Should schools be deemed low- or high-performing on the basis of failure on one
indicator or a combination of multiple indicators? If the latter, what rules should be
used to combine indicators in a fair and appropriate manner?
The position taken in this paper is that multiple indicators of school performance
are needed to fairly and accurately categorise levels of performance. This article
describes how publicly available school-level data can be analysed in order to assess
the relative performance levels of New Zealand secondary schools. A set of
combination rules was developed to categorise schools as either high-, mid-, or
relatively low-performing. In addition improving schools were identified. The analysis
involved comparing schools-of-similar-type using five different indicators based on
student results from 2004 to 2007 on the National Certificate in Educational
Achievement (NCEA); an achievement system that is characterised by its close
assessment of the taught curriculum.
The first section reviews the literature to outline six possible methods of assessing
school performance.The second section describes our preferredsystem for using school
performance data to identify improving schools and those in the high-, mid-, or
low-performing categories. The third section then provides background on the New
Zealand qualification system so that the rationale for the method of assessing relative
school performance in the fourth sectionis clear to the reader. Finally, findingsusing this
method are reported and the advantages and limitations of this approach are discussed.
Methods of assessing and comparing school performance
At least six methods of assessing and comparing school performance are used in
school effects research:
(1) gross productivity;
(2) value-added;
(3) regression discontinuity;
(4) growth curve analysis;
JEA
49,4
434
To continue reading
Request your trial