Approaches to the validation of quality frameworks for e‐learning
| Published date | 26 September 2008 |
| DOI | https://doi.org/10.1108/09684880810906490 |
| Date | 26 September 2008 |
| Pages | 347-362 |
| Author | Alistair Inglis |
| Subject Matter | Education |
Approaches to the validation of
quality frameworks for e-learning
Alistair Inglis
Victoria University, Melbourne, Australia
Abstract
Purpose – This paper aims to compare the ways in which a range of quality frameworks have been
validated and to identify a number of factors that have an impact on validation processes.
Design/methodology/approach – Seven examples of published quality frameworks applicable to
the field of e-learning are described and the methods used to validate each of the frameworks are
identified and compared. The article concludes by considering a range of factors that have the
potential to have an impact on such validation processes.
Findings – Six methods of validation were found to have been used in relation to development of the
seven frameworks that were examined: reviewing the research literature related to effectiveness in
online learning; seeking input from an expert panel; undertaking empirical research; undertaking
survey research; conducting pilot projects; and drawing on case studies. From the variety of
approaches used and the ways in which they were used it was concluded that a recognised set of
procedures for validation of quality frameworks has not yet emerged.
Research limitations/implications – The most important limitation of this study is that its
findings are dependent on the particular quality frameworks selected for inclusion.
Practical implications – The paper draws attention to the need for more attention to be paid to the
development of methods of validation that are both objective and robust.
Originality/value – No previous studies were located that have looked specifically at the processes
used to validate quality frameworks. This paper therefore provides some initial baseline data upon
which to base future work.
Keywords Quality, E-learning,Modelling
Paper type Research paper
Introduction
Concern amongst practitioners in the field of e-learning about the issue of quality has
grown in recent years (McLoughlin and Visser, 2003; Oliver, 2005). Concern about
quality in e-learning has also grown amongst education and training providers and
national accreditation and quality agencies (Quality Assurance Agency, 2004; Western
Cooperative for Educational Telecommunication, 2002). Concerns about quality are
being manifested as initiatives to implement processes for assuring that minimum
standards are being met, and that an overall improvement in the quality of courses
offered online will be achieved over time.
When educators think of quality they tend to want to arrive at a single global
measure. This is a fruitless quest. Quality subsists in a variety of properties and a
course’s quality cannot be measured along a single dimension from high to low. One
cannot measure the quality of an educational program in the same way as one might
measure the quality of the delivery of a customer service by a commercial service
provider. Neither can a course’s quality be ranked along a single dimension. The
unidimensional approach to measuring quality overlooks the fact that the delivery of
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0968-4883.htm
The validation
of quality
frameworks
347
Received April 2008
Revised May 2008
Accepted June 2008
Quality Assurance in Education
Vol. 16 No. 4, 2008
pp. 347-362
qEmerald Group Publishing Limited
0968-4883
DOI 10.1108/09684880810906490
courses is a multi-faceted activity and that the process of measuring quality in
education relies on unpacking the range of factors that impact the learner’s experience
and measuring these separately.
To bring together the various elements that contribute to the quality of a course,
most quality processes rely on the use of a quality framework of one type or other. A
quality framework defines the set of variables in terms of which quality is measured
and the way in which it is measured. Examples of quality frameworks include the
quality improvement framework described by Inglis et al. (2002), the benchmarking
framework described by McKinnon et al. (2000), and the eLearning benchmarks
produced by Phipps and Merisotis (2000).
Quality frameworks are conceptual structures used to identify the range of factors
considered important to decisions in relation to quality. However, the concept of a
quality framework is not well defined – there are no sets of descriptors that prescribe
the form that a quality framework ought to take. Quality frameworks are used to
underpin all four quality processes described above.
The value of a quality framework depends upon the way in which it has been
constructed. While there is a substantial literature examining the factors that should be
considered in judging quality in relation to courses offered online (McLoughlin and
Visser, 2003), little attention has been given to the evaluation of the importance of these
factors themselves. While one can construct a plausible quality framework purely in
the basis of intuition, the value of the framework for measuring quality will depend
on the correspondence between the elements of the framework and the factors that
impact the effectiveness with which the students learn. What is needed is a way of
certifying the adequacy of the framework. Yet, there does not appear to be an accepted
approach to certifying the quality frameworks themselves. The users of quality
processes seldom give thought to the strength of the frameworks upon which these
processes rely. Judgements of quality are made against a set of criteria. However, if the
criteria are flawed, then the judgements of quality that rely on those criteria will
themselves be flawed. Failure to consider this factor can result in unwarranted reliance
being placed on factors for which there is no underpinning empirical support. The set
of criteria used in quality processes are embedded in or based on the particular quality
framework that is used. The judgements that are made in relation to quality when a
quality framework is being used therefore depend very much on factors that impact the
adequacy of the framework: the elements from which a framework is constructed, the
way in which the elements go together, and the way in which the framework is used in
practice.
The literature on quality processes in education and on the construction of quality
frameworks has very little to say about the validation. There seems to be a
taken-for-granted assumption that if the originator of a framework has thought
sufficiently about the development and delivery of courses appropriately, then this will
suffice to assure its validity. However, this confidence in the omnipotence of individual
contributors to the literature appears to be misplaced. People attend to different aspects
of course delivery. What may be considered important by one person may not be
considered important by another. Yet even if the need for undertaking some form of
validation is recognised, there is still the question of how that should be undertaken.
This situation is probably a reflection of the fact that e-learning is an emerging field,
and the development of quality processes in relation to e-learning is developing fast. In
QAE
16,4
348
To continue reading
Request your trial