Quality assurance of multiple‐choice tests

Published date01 October 2006
Pages398-404
DOIhttps://doi.org/10.1108/09684880610703974
Date01 October 2006
AuthorMartin E. Bush
Subject MatterEducation
Quality assurance of
multiple-choice tests
Martin E. Bush
London South Bank University, London, UK
Abstract
Purpose – To provide educationalists with an understanding of the key quality issues relating to
multiple-choice tests, and a set of guidelines for the quality assurance of such tests.
Design/methodology/approach – The discussion of quality issues isstructured to reflect the order
in which those issues naturally arise. It covers the design of individual multiple-choice questions,
issues relating to the question bank as a whole, choice of test format, and what can be learned through
post-test analysis. The paper offers practical advice, with an emphasis on maximising test reliability.
Findings – It is recognised that considerable expertise and effort is required to undertake a thorough
post-test statistical analysis, but pre-test quality assurance is relatively straightforward, if
labour-intensive. The question of which is best amongst the various alternative test formats is left
open.
Originality/value – The general issue of quality assurance of multiple-choice tests is surely an
important one, yet the author is not aware of any other publication that deals directly with this topic.
Keywords Scoring procedures(tests), Quality assurance, Assessment,Education
Paper type General review
Introduction and terminology
Multiple-choice tests are already in widespread use within many areas of education,
and their importance as a method of assessment seems likely to grow further with the
advent of e-learning. They are attractive for a variety of reasons. For example, they can
be used to test higher-order thinking skills in addition to knowledge recollection, there
is no dependence on good writing skills, and a wide range of topics can be covered in
one sitting. Also, the marking is easily automated – which makes them idea l for
e-learning.
Designing good test questions is not easy, however. The strain to produce a large
question bank can easily lead to poorly written and/or unintentionally overlapping
questions. In this paper we review a range of quality issues relating to multiple-choice
tests, and I offer practical advice regarding quality assurance both of a questio n bank
and the tests that are derived from it, with an emphasis on maximising test reliability
(defined below).
As far as quality assurance is concerned, too often the only considerations are:
(1) Do the questions look reasonable bearing in mind the subject matter?
(2) Does the distribution of marks look reasonable bearing in mind the performance
of the examinees in other examinations?
The first question should be considered before any testing takes place. The second
question can only be considered after the testing has taken place. Given additional
effort, quality assurance can go a lot further – as this article explains.
Conventionally, an “n-choice test” is made up of a number of questions each with a
question stem, one correct answer and n21 distractors (incorrect answers). We can
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0968-4883.htm
QAE
14,4
398
Quality Assurance in Education
Vol. 14 No. 4, 2006
pp. 398-404
qEmerald Group Publishing Limited
0968-4883
DOI 10.1108/09684880610703974

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT