Website quality evaluation: a model for developing comprehensive assessment instruments based on key quality factors
| DOI | https://doi.org/10.1108/JD-11-2022-0246 |
| Published date | 28 February 2023 |
| Date | 28 February 2023 |
| Pages | 95-114 |
| Subject Matter | Library & information science,Records management & preservation,Document management,Classification & cataloguing,Information behaviour & retrieval,Collection building & management,Scholarly communications/publishing,Information & knowledge management,Information management & governance,Information management,Information & communications technology,Internet |
| Author | Alejandro Morales-Vargas,Rafael Pedraza-Jimenez,Lluís Codina |
Website quality evaluation:
a model for developing
comprehensive assessment
instruments based
on key quality factors
Alejandro Morales-Vargas
Department of Social Communication, University of Chile, Santiago, Chile, and
Rafael Pedraza-Jimenez and Llu
ıs Codina
Department of Communication, Pompeu Fabra University, Barcelona, Spain
Abstract
Purpose –The field of website quality evaluation attracts the interest of a range of disciplines, each bringing
its own particular perspective to bear. This study aims to identify the main characteristics –methods,
techniques and tools –of the instruments of evaluation described in this literature, with a specific concern for
the factors analysed, and based on these, a multipurpose model is proposed for the development of new
comprehensive instruments.
Design/methodology/approach–Following a systematicbibliographic review, 305 publications on website
qualityare examined, the field’s leading authors, their disciplines of origin and the sectors to which the websites
being assessed belong are identified, and the methods they employ characterised.
Findings –Evaluations of websitequality tend to be conducted with one of threeprimary focuses: strategic,
functional or experiential. The technique of expert analysis predominates over user studies and most of the
instruments examined classify the characteristics to be evaluated –for example, usability and content–in to
factorsthat operate at differentlevels, albeit that thereis little agreement on the namesused in referring to them.
Originality/value –Based on the factors detected in the 50 most cited works, a model is developed that
classifies these factors into 13 dimensions and more than 120 general parameters. The resulting model provides
a comprehensive evaluation framework and constitutes an initial step towards a shared conceptualization of
the discipline of website quality.
Keywords Website quality, Website, Website evaluation, Expert analysis, Heuristic evaluation,
User experience, Usability
Paper type Research paper
Introduction
Over the last three decades, websites have become one of the most important platforms on the
Internet for disseminating information and providing services to society. Shortly after their
first appearance, the need to evaluate website quality became evident. The earliest analyses
were developed by experts in human-computer interaction and comprised usability heuristics
Website
quality
evaluation
95
© Alejandro Morales-Vargas, Rafael Pedraza-Jimenez and Llu
ıs Codina. Published by Emerald
Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0)
licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both
commercial and non-commercial purposes), subject to full attribution to the original publication and
authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/
legalcode
Funding: This work is supported by Spain’s Ministry of Science and Innovation (MICINN) project
“Parameters and strategies to increase the relevance of media and digital communication in society:
curation, visualisation and visibility (CUVICOM)”. PID 2021-123579OB-I00.
The current issue and full text archive of this journal is available on Emerald Insight at:
https://www.emerald.com/insight/0022-0418.htm
Received 17 November 2022
Revised 19 January 2023
Accepted 25 January 2023
Journal of Documentation
Vol. 79 No. 7, 2023
pp. 95-114
Emerald Publishing Limited
0022-0418
DOI 10.1108/JD-11-2022-0246
(Nielsen, 2020) and design principles (Tognazzini, 2014) and rules (Shneiderman, 2016), aimed
at improving interfaces. In parallel, inspection of the technical specifications of websites and
the verification of standards for application development emerged (W3C, 2016).
Subsequently, interest has grown in designing for an optimal user experience (Garrett,
2011) and quantifying user experience perceptions (Sauro and Lewis, 2016).
This evolution highlights the fact that, from its very outset, website quality evaluation has
taken different approaches, analysing a range of different characteristics and employing a
variety of methodologies. This may well be an indication that the discipline of website quality
has yet to be fully consolidated and that its field of study is not readily delimited. This
conclusion is further strengthened by the fact that the field has yet to agree on a formal
definition for itself (Law et al., 2010;Semer
adov
a and Weinlich, 2020).
Over the last twenty years, a number of different authors have offered their definitions.
Leavitt and Shneiderman (2006), in one of the earliest attempts, define website evaluation as
the act of establishing a correct and exhaustive set of user requirements, ensuring a site
provides useful content that meets user expectations, while setting usability goals. For
Aladwani and Palvia (2002), website quality is determined primarily by the degree to which a
website’s features are perceived by users to meet users’needs and to reflect the site’s overall
excellence; while for Gregg and Walczak (2010) website quality constitutes the attributes that
contribute to its usefulness to consumers. Most recently, “the ability of a website to meet the
expectations of its users and owners, as determined by a set of measurable attributes”is the
definition proposed by Morales-Vargas et al. (2020, p. 3).
While these definitions coincide in the need for websites to satisfy user expectations, they
differ in terms of the factors that should come under examination. Indeed, as an emerging
research area, website quality in the literature has yet to achieve a common operationalization
and each study tends to highlight different measures that are relevant to its own particular
context (Law, 2019). When evaluating the quality of a website, it is important to know what
can be measured and how to measure it (Akincilar and Dagdeviren, 2014). On the other hand,
the evaluation of website quality is not the same as undertaking a traditional quality
evaluation since it involves multi-criteria decision-making (Ecer, 2014), making it a
particularly complex activity.
Thus, it is critical to identify the factors and characteristics that should be evaluated.
In this regard, we can identify a first traditional focus to the question that might be defined as
functional. Here, the focus is on the inspection of a website’s inherent characteristics,
including its content, information architecture and visual design, as well as its technical and
operational features, linked to technology and security (ISO, 2008;Leung et al., 2016). The
second approach, which we can define as experiential, focuses on user experience and
perceptions and examines such factors as usability, accessibility, satisfaction and interaction
(Maia and Furtado, 2016;Tullis and Albert, 2013). A third approach is more strategic in
nature, focusing as it does on meeting the site owner’s objectives, and on the use of
performance, visibility and positioning metrics, among others (Kaur and Gupta, 2014;
Sanabre et al., 2020).
All evaluation instruments of website quality, regardless of their particular focus, have in
common the fact that they seek to conceptualise and delimit the target they seek to measure
using some type of unit. The literature employs different names for these units, be it
attributes, characteristics, variables (ISO, n.d.), factors or criteria (Chiou et al., 2010). Their use
is largely synonymous, being terms that allude to the distinctive features of a certain property
of the analysed entity, that is, websites. Codina and Pedraza-Jim
enez (2016) propose
addressing these properties, from the most general to the most specific, as dimensions,
parameters and indicators, a terminology that we employ in this article. Dimensions
constitute the generic properties of a website that we might want to evaluate. These can be
divided, in turn, into more specific units, referred to as parameters. Finally, the indicators are
JD
79,7
96
To continue reading
Request your trial