Performance assessment for school leaders: comparing field trial and implementation results

Pages38-59
Published date23 August 2019
Date23 August 2019
DOIhttps://doi.org/10.1108/JEA-05-2018-0087
AuthorMargaret Terry Orr,Liz Hollingworth,Barbara Beaudin
Subject MatterEducation,Administration & policy in education,School administration/policy,Educational administration,Leadership in education
Performance assessment for
school leaders: comparing field
trial and implementation results
Margaret Terry Orr
Department of Educational Leadership, Administration and Policy,
Fordham University, New York, New York, USA
Liz Hollingworth
College of Education, University of Iowa, Iowa City, Iowa, USA, and
Barbara Beaudin
Independent Researcher, Avon, Connecticut, USA
Abstract
Purpose The purpose of this paper is to compare two years of results for one states performance-based
assessments for principal licensure Performance Assessment for Leaders (PAL). This includes the field trial
(20142015) and first year of statewide implementation (20152016) when passing score requirements and
fees were added. Survey results on candidatescareer aspirations provide concurrent validation.
Design/methodology/approach Two years of PAL submissions (n¼569 candidates) were scored by
trained, certified scorers. Task and total score results were compared by year, preparation pathway and
gender. Online feedback survey results on career aspirations for (n¼146 candidates) were compared by year.
Findings The results show that PAL assessments measure independent dimensions of leadership,
differentiate candidates on leadership knowledge and skills, and confirm PALs internal validity.
Implementation year scores were higher than field trial scores, and preparation program candidates scored
better than non-program candidates did. Candidate career aspirations were stronger in the implementation
year than during the field trial.
Research limitations/implications The study is limited to one states candidates, but findings are
generalizable based on the wide range among candidatesdistricts (demographically and economically).
Practical implications The results are promising for the leadership preparation and assessment field,
demonstrating the effectiveness of performance assessment for authentic evaluation of leadership candidates
knowledge and skill and overall readiness for initial leadership work.
Originality/value This is the first large scale performance assessment for aspiring leaders designed for
state licensure decisions. It is being replicated in another state and shown promise for both formative and
summative leadership assessment.
Keywords Performance assessment, Licensure, Aspiring leaders
Paper type Research paper
Introduction
This paper provides evidence on the validity and use of a new performance assessment
system currently used in one state Massachusetts to evaluate candidate readiness for
leadership licensure and initial school leadership positions. Two years of candidate
assessments for work products submitted for four performance tasks, each scored using
several standards-based rubrics, enabled exploration of measurement validity, the
consequential use of passing scores, and the effects of the imposition of fees on candidate
assessment participation and performance. Assessment results also allowed for comparison
Journal of Educational
Administration
Vol. 58 No. 1, 2020
pp. 38-59
© Emerald PublishingLimited
0957-8234
DOI 10.1108/JEA-05-2018-0087
Received 12 May 2018
Revised 8 November 2018
3 April 2019
5June2019
1July2019
21 July 2019
Accepted 25 July 2019
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0957-8234.htm
This research was made possible by contract support from the Massachusetts Department of
Elementary and Secondary Education. Copies of the PAL technical report, Massachusetts Performance
Assessment for Leaders (PAL) Technical Report Summary of Validity and Reliability Studies for
20142015 Field Trial of PAL, are available from the department or from the authors. Correspondence
concerning this research should be addressed to Margaret Terry Orr (morr4@fordham.edu)
38
JEA
58,1
of year-to-year differences in the relationship between scores and candidate characteristics,
and of modes of leadership preparation. These results provide critical guidance in
addressing the current policy need for valid and reliable authentic leadership assessments
and the consequences of their use.
The need for effective leadership assessments
There has been a persistent need among states and local areas for valid, high quality school
leader readiness assessments as a means of improving school leader quality and
effectiveness. In the USA, states strive to insure the high quality of leaders in part through
standard setting and licensure policies (Manna, 2015; Shelton, 2012). While appropriate
standards establish expectations for school leader effectiveness, there are few available
mechanisms for determining the extent to which aspiring leaders are ready for an initial
leadership position.
Typically, state school leader licensure policies directly or indirectly set expectations for
leadership preparation. To determine whether a license is granted, many states accept
measures of leader knowledge as evidence of readiness (Hackmann, 2016). Commonly used,
primarily knowledge-based measures that evaluate candidatesleadership readiness include
the successful completion of graduate coursework, a degree in educational leadership, and a
state-required content knowledge test (Davis, 2010; Grissom et al., 2017; Hackmann, 2016).
While varying widely in their state licensure requirements (Davis, 2010), most states
adopted the Interstate School Leaders Licensure Consortium (ISLLC) standards in whole or
in a modified form for preparation and certification decisions (Hackmann, 2016; Shelton,
2012). Given the insufficiency of knowledge- or cognitive-related measures, policy analysts
in recent years have advocated for better licensure assessments to determine leadership
candidate quality and improve readiness (Shelton, 2012).
In addition, federal educator evaluation policies in the early 2000s required states and
districts to develop principal evaluation systems in order to identify highly effective leaders
(as was similarly required for teachers) and, by extension, to determine candidate readiness
for leadership licensure (US Department of Education, 2010). The federal grant program,
Race to the Top (RTTT), provided funds for states to invest in the development of
assessments and enacted other policy changes to improve teacher and leader effectiveness.
The emergence of performance assessments to fill the need
Performance assessments are a promising approach to determining leadership quality and
readiness in public education, and they speak to the widely expressed need for more
effective evaluation measures.
As one of the states to receive RTTT funding in 2011, Massachusetts decided to test this
approach. Specifically, the state engaged a six-member team of national experts on
leadership and performance assessments to develop a performance assessment system,
called the Performance Assessment for Leaders (PAL), with mandated participation for
leadership candidates. PAL, through the establishment of performance expectations, would
improve and then measure candidatesreadiness for principal licensure. This candidate
assessment requirement was implemented simultaneously with revised requirements for
leadership preparation for the states multiple university and non-university-based
pathways. The new requirements were intended to foster the certification of a large number
of candidates[1] in a variety of both district settings (from very rural to large city) and
leadership preparation pathways: a state-approved preparation program, an administrative
apprenticeship/internship option, and a panel review option (Orr, Pecheone, Snyder,
Murphy, Palanki, Beaudin et al., 2018).
PAL was created as a set of four inter-related performance assessment tasks for which
candidates complete field-based work to demonstrate their leadership knowledge and skills.
39
Performance
assessment for
school leaders

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT