Improving Performance Appraisal by Using Multiple Appraisers
| Pages | 13-16 |
| Date | 01 July 1981 |
| DOI | https://doi.org/10.1108/eb057199 |
| Published date | 01 July 1981 |
| Author | Mark R. Edwards |
| Subject Matter | Economics,Information & knowledge management,Management science & operations |
Improving Performance
Appraisal by Using Multiple
Appraisers by Mark R. Edwards*
Abstract
Improvement in performance appraisal systems are neces-
sary because current methods are neither accurate nor
fair. Recent attempts to improve performance appraisals
by training raters has been shown to have the opposite
effect-actually decreasing the accuracy of the perfor-
mance measures. Multiple rater performance appraisal
systems (MRPAS) provide an opportunity to increase
appraisal accuracy by combining judgements of individual
raters.
However, before MRPAS are used, numerous
objections must be overcome including system acceptance,
finding raters, system design, desire to participate, and
the potential use of too much management time. The
Objective Judgement Quotient (OJQ) performance
appraisal process is used as a demonstration method that
has developed solutions to each of these objections. When
MRPAS are used effectively, the fairness and usefulness
of multiple rater systems compared to single rater systems
are so clearly superior that users of MRPAS are likely to
adopt the view that single rater systems are obsolete.
Each individual has a
personal view of the world
Are single rater performance appraisals obsolete? Pos-
sibly, for users of multiple rater systems.
Traditional performance appraisal systems are not
accurate [1]. Current performance measures too often
do not correlate highly with true performance and there-
fore,
are unfair.
Each individual has a personal view of the world.
This individualism is reflected in the variation that
occurs when observation ratings are compared. Borman
has demonstrated the dramatic variation among indi-
vidual raters when the raters are viewing the same
performance [2]. Bernardin and Pence, as well as Bor-
man, have shown that, contrary to popular
belief,
rater
training does not improve rater accuracy in performance
appraisal [3].
Another approach to improving rating accuracy is by
combining several judgements (each of which may be
inaccurate). The consensus of the multiple judgements
will provide a better estimate of true performance than
the traditional individual judgement. However, a recent
national survey of 293 employers for the Conference
Board did not report any firms using a multiple rater
performance appraisal system [4].
Since individual ratings of performance are subject to
extreme biases and are not accurate, why have emp-
loyers not developed multiple rater performance
appraisal systems? The objections are many and
include:
–Our personnel are accustomed to single supervisor
evaluations and would not accept a multiple rater
system.
–Where would we find enough raters?
–How would we design a multiple rater system?
–People do not like to do appraisals now. Multiple
appraisers would mean requiring more people to be
involved in the appraisal process.
–Our single rater evaluation system takes too much
time.
We would not want to increase the total evalu-
ation time.
This article offers solutions that many organisations
have developed to overcome each of these objections.
A Multiple Rater System
A multiple rater performance appraisal system
(MRPAS) should meet or exceed the requirements of
single rater systems, including:
(1) Be perceived as fair by ratees and raters.
(2) Meet EEO requirements (provide documentation,
comparability, equity, and be culturally fair).
(3) Be cost effective.
(4) Be easy to use and require little management time.
(5) Provide useful quantified feedback for employees,
management, and appraisers.
(6) Be as objective as possible in comparing relative
performance.
(7) Minimise biases such as leniency and halo and be
inflation-proof.
The Objective Judgement Quotient process was
designed to meet these requirements [5].
The OJQ System
OJQ is a measurement method that uses multiple raters.
It provides a simple, scaled rank order numerical solu-
tion when judgement data is combined. The basic OJQ
process has been described by Bartlett and Linden [5]
and is used here to demonstrate features and benefits of
MRPAS.
The OJQ system has six unique features
*Mark R. Edwards, DBA, is an assistant professor at Arizona State
University. His industrial experience includes service as Manager of
Personnel for The Greyhound Corporation and extensive consulting
on performance appraisal, compensation, productivity improvement,
and human resources planning.
JULY/AUGUST 1981 13
To continue reading
Request your trial