Towards a holistic model for quality of learning object repositories. A practical application to the indicator of metadata compliance
| Date | 02 October 2017 |
| Published date | 02 October 2017 |
| DOI | https://doi.org/10.1108/EL-10-2015-0202 |
| Pages | 953-976 |
| Author | Christian Vidal-Castro,Alejandra Andrea Segura Navarrete,Victor Menendez-Dominguez,Claudia Martinez-Araneda |
| Subject Matter | Information & knowledge management,Information & communications technology,Internet |
Towards a holistic model for
quality of learning
object repositories
A practical application to the indicator of
metadata compliance
Christian Vidal-Castro and Alejandra Andrea Segura Navarrete
Sistemas de Información, Universidad del Bío-Bío, Concepción, Chile
Victor Menendez-Dominguez
Facultad de Matemáticas, Universidad Autónoma de Yucatán, Mérida, México, and
Claudia Martinez-Araneda
Departamento de Ingeniería Informática,
Universidad Católica de la Santísima Concepción, Concepción, Chile
Abstract
Purpose –This paper aims to address the need to ensure the quality of metadata records describing
learning resources. We propose improvements to a metadata-qualitymodel, specifically for the compliance
sub-feature of the functionality feature. Compliance is defined as adherence level of the learning object
metadatacontent to the metadata standard used for its specification. The paper proposesmetrics to assess the
compliance, which are applied to a set of learning objects, showing their applicability and usefulness in
activitiesrelated to resources management.
Design/methodology/approach –The methodology considers a first stage of metrics refinement to
obtain the indicator of the sub-feature compliance. The next stage is the proposal evaluation, where it is
determined if metrics can be used as a conformity indicator of learning object metadata with a standard
(metadata compliance). The usefulness of this indicator in the information retrieval area is approached
through an assessment of learning objectswhere the quality level of its metadata and the ranking in which
they are retrievedby a repository are correlated.
Findings –This study confirmed that the best results for metrics of standardization, completeness,
congruence, coherence, correctness and understandability, which determine the compliance indicator, were
obtained for learning objectswhose metadata were better labelled. Moreover, it was foundthat the learning
objects with the highest levelof compliance indicator have better positions in the rankingwhen a repository
retrievesthem through an exact search based on metadata.
Research limitations/implications –In this study, only a sub-feature of the qualitymodel is detailed,
specificallythe compliance of learning object standard. Another limitationwas the size of the learning objects
set used in the experiment.
Practical implications –This proposal is independent fromany metadata standard and can be applied
to improve processes associated with the management of learning objects in a repository-like retrieval and
recommendation.
This paper is the result of work by the SOMOS research group (SOftware –MOdelling –Science),
funded by the Dirección de Investigación and Facultad de Ciencias Empresariales of the Universidad
del Bío-Bío, Chile. The authors thank the Facultad de Ingeniería de la Universidad Católica de la
Santísima Concepción, Chile.
Learning
object
repositories
953
Received14 October 2015
Revised20 October 2016
Accepted4 December 2016
TheElectronic Library
Vol.35 No. 5, 2017
pp. 953-976
© Emerald Publishing Limited
0264-0473
DOI 10.1108/EL-10-2015-0202
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0264-0473.htm
Originality/value –The originality and value of this proposal are related to quality of learning object
metadata consideredfrom a holistic point of view through six metrics. Thesemetrics quantify both technical
and pedagogical aspects through automatic evaluation and supported by experts. In addition, the
applicability of the indicatorin recovery systems is shown, by example to be incorporated as an additional
criterionin the learning object ranking.
Keywords Metadata, Digital repositories, Quality, IEEE-LOM, Learning object
Paper type Research paper
Introduction
Quality assessment is a complexissue in any scope. In this paper, the focus is on the quality
of the materials used in e-learning in order to supportthe teaching-learning process. In this
sense, the notion of a learning resource has been formalized in the concept of a learning
object (LO). In this research, the definition proposed by McGreal (2004, p. 21) is used: a
learning object is “any reusable digital resource that is encapsulated in a lesson or
assemblage of lessons grouped in units, modules, courses, and even programmes”.
Physically, a learning object can be considered as a package that contains a learning
resource, a metadata and a manifestdescribing the package contents (IMS GLC, 2007).
Metadata standards are key elements in this context. Metadata standards are formal
specifications used to describe educational resources (Al-Khalifa and Davis, 2006). IEEE-
LOM (Learning Technology Standards Committee (LTSC), 2002) developed metadata
standards designed specifically for learning objects, and defined the syntax and semantics
of metadata. They consist of more than 60 descriptors grouped intoa conceptual scheme of
nine categories: general, life cycle, meta-metadata, technical, educational, rights, relation,
annotation and classification. Using a standard or application profile for the labelling of
learning objects is not necessarilyan indicator of quality, although contributes to the objects
correct description.
This paper proposes improvements to the model presented by Vidal et al. (2008),
specifically the sub-characteristic for compliance belonging to the functionality
characteristic. Compliance is defined as the adherence level of the learning object metadata
content to the metadata standard used for its specification. Two research questions have
been defined that guide thiswork:
RQ1. Does a learning object with metadata properlylabelled according to standard get
higher values in the proposed compliancemetrics?
RQ2. Is a learning object with the highest levelof quality more likely to be recovered in
a repository by exact searching?
To answer both questions, two experimentsare presented. The first one is designed with the
aim of validating the metrics that make up the indicator of compliance quality. The second
is designed with the aim of verifying, in a case study,that there is a correlation between the
quality of the learning objectmetadata and the chances of recovering it in a repository.
As shown in Figure 1, the methodology is based on the model proposed by Vidal et al.
(2008). First, a refining step isperformed where the characteristics and sub-characteristicsof
the complete model are defined. In the next stage, metrics to gauge the compliance of sub-
characteristic are defined.The next steps consider the evaluation of the proposal. In the first
experiment, these indicators are calculated for a set of labelled learning objects. Then, the
metadata quality levels of each object and the position in which they are recovered by an
exact searching systemare correlated.
EL
35,5
954
To continue reading
Request your trial