Academics’ feedback on the quality of appraisal evidence
| DOI | https://doi.org/10.1108/QAE-05-2014-0023 |
| Published date | 06 July 2015 |
| Date | 06 July 2015 |
| Pages | 279-294 |
| Author | Chenicheri Sid Nair,Jinrui Li,Li Kun Cai |
| Subject Matter | Education,Curriculum, instruction & assessment,Educational evaluation/assessment |
Academics’ feedback on the
quality of appraisal evidence
Chenicheri Sid Nair
Centre for Advancement of Teaching and Learning,
The University of Western Australia, Perth, Australia
Jinrui Li
Applied Linguistics, University of Waikato, Hamilton, New Zealand, and
Li Kun Cai
Foreign Language Department,
North China University of Science and Technology, Tangshan, China
Abstract
Purpose – This paper aims to explore academics’ perspectives on the quality of appraisal evidence at
a Chinese university.
Design/methodology/approach – An online survey with both closed items and open-ended questions
was distributed among all academics at the university (n⫽1,538). A total of 512 responded to the
questionnaire. The closed items were initially analysed using EXCEL and SPSS; the open-ended questions
were thematically analysed.
Findings – The academics believed that the quality of student survey and peer observation of
teaching were affected by subjectivity and the lack of understanding of appraisal. Academics also
suggested that appraisals should be contextualised and the approach standardised. The study suggests
the need for training that informs and engages relevant stakeholders to ensure the rigour of appraisal.
Originality/value – The study raises the issue of quality assurance regarding appraisal data from the
perspective of academics. It is based on the collaborative effort of academics in Australia, China and
New Zealand, with the support of the management staff at the case study university. The study informs
both appraisers and academics of quality assurance issues in appraisal. It also contributes to the
literature, in that it initiates dialogues between communities of practices through collective questioning
on the quality and mechanisms of appraisal in tertiary education.
Keywords Training, Performance appraisal, Quality assessment, Student survey, Education,
Questionnaire, Quality of evidence, Peer observation of teaching, Teacher appraisal
Paper type Research paper
Introduction
Staff appraisals are not only used as indicators for nancial accountability, university
ranking and quality assurance (Shin, 2011), but are also related to personnel decisions,
promotion and professional development (Alderman et al., 2012;Marsh, 2007;Minelli
et al., 2007). This nexus of demands and purposes in tertiary education (Pinheiro, 2013)
results in the challenge of denition, collection and interpretation of performance
evidence for appraisal.
Trigwell (2011) argues that it is a challenge to select reliable indicators for appraisals.
Egginton (2010) goes on further to argue that it not easy to make fair judgement on good
performance in appraisals, as it often involves conicting goals and values. For
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0968-4883.htm
Quality of
appraisal
evidence
279
Received 27 May 2014
Revised 11 September 2014
Accepted 16 October 2014
QualityAssurance in Education
Vol.23 No. 3, 2015
pp.279-294
©Emerald Group Publishing Limited
0968-4883
DOI 10.1108/QAE-05-2014-0023
example, although good teaching is often related to effective and meaningful teaching
that results in learning (Casey et al., 1997), its interpretations vary within and across
disciplines (Kreber, 2002) and in different contexts. Minelli et al. (2007) further dene
four elements that bring about organisational impact of appraisals: the idea of the
assessment, the method of collection and analysis, the bodies that look after the
evaluation process and the way such data are used in the institution.
Generally speaking, research publications and student appraisal of teaching are key
evidential instruments used in university staff appraisal. However, compared with
teaching, research achievement seems more important to academics’ career
development (Taylor, 2007). In fact, academics often suffer from the tension between
teaching and research, in that teaching speciality is often under-valued (Bexley et al.,
2013). The same tension is felt among academics in universities in China (Du et al., 2010).
There is also a trend towards collecting evidence by means of qualitative approaches,
such as student interviews and peer observation of teaching, to reect holistically staff’s
achievement (Bennett and Nair, 2010). However, little is known about whether, and how,
the quality of such evidence data is ensured. A numbers of studies on course appraisal
via student survey and/or peer review, and academics’ feedback on the quality of
appraisal evidence collected by student survey and class observation by colleagues and
experts, have been carried out in many Western countries (Stein et al., 2013). Hence, the
purpose of this study is to explore effects and issues of appraisal in a top provincial-level
university in north China, from the perspectives of Chinese academics. The particular
focus of this article is academics’ comment on the quality of the three common indicators
of performance: student’ survey, peer observation of teaching and research.
This study is a collaborative research project carried out by a community of
teacher-researchers and teacher development experts from Australia, China and New
Zealand, with support from the management staff of the case study university. It
contributes to the existing literature, in that it adds to the dialogue between communities
of practice through collective questioning of the existing mechanisms of appraisal in
tertiary education.
Student appraisal of teaching
One controversial form of evidence used in teaching appraisal is the end-of-course
student survey and/or interview. Regarding the validity and reliability of the student
survey, Chen and Watkins (2010), through analysis of the scores on 435 teachers in two
semesters and survey data among 388 of the teachers in a university in China, found that
the students’ appraisals were consistent and valid. This research supports previous
research supporting the reliability and validity of student appraisals (Benton and
Cashin, 2012;Marsh, 2007;Stein et al., 2013;Stowell et al., 2012). However, studies by
Crumbley et al. (2001),Pounder (2007),Buchanan (2011) and Darwin (2012) challenge
assumptions underpinning appraisals in higher education and question not only the
ethics but the validity of higher education institutions’ reliance on student surveys as
measures of effective pedagogic practice.
Some of the issues surrounding the validity and reliability of students’ appraisal
reported include teachers’ workload, their rapport with students, grading of
students’ assignments (Boysne, 2008;Amin, 2002), teachers’ work priorities, the
nature of courses and disciplines and students’ interest in the course (Rantanen,
2013). Comparatively, grading also seems to be a very important factor that affects
QAE
23,3
280
To continue reading
Request your trial