Heuristic evaluation of e-dictionaries
| DOI | https://doi.org/10.1108/LHT-07-2017-0144 |
| Date | 18 June 2018 |
| Published date | 18 June 2018 |
| Pages | 319-338 |
| Author | Liezl H. Ball,Theo J.D. Bothma |
| Subject Matter | Library & information science,Librarianship/library management,Library technology,Information behaviour & retrieval,Information user studies,Metadata,Information & knowledge management,Information & communications technology,Internet |
Heuristic evaluation of
e-dictionaries
Liezl H. Ball and Theo J.D. Bothma
Department of Information Science, University of Pretoria, Pretoria, South Africa
Abstract
Purpose –The purpose of this paper is to discuss the heuristic evaluations of five e-dictionaries according to
the criteria developed by Ball and Bothma (2018). E-dictionaries are increasingly making use of modern
information technology to create advanced information tools. It is necessary to ensure that these new
products are still usable. Heuristic evaluation is a usability evaluation method used to evaluate the usability
of a product.
Design/methodology/approach –Five e-dictionaries were evaluated through heuristic evaluation.
This method requires an evaluator to evaluate a product by using a set of criteria or guidelines.
Findings –Various usability issues, as well as good features of e-dictionaries, could be identified through
these evaluations, and are discussed under the categories of content, information architecture, navigation,
access (searching and browsing), help, customisation and use of other innovative technologies.
Originality/value –Through the evaluations in this study, the criteria could be validated and an example of
how the criteria can be used to evaluate e-dictionaries could be presented.
Keywords Evaluation criteria, Heuristic evaluation, E-dictionaries, Electronic dictionaries, Usability evaluation,
Expert evaluation
Paper type Research paper
1. Introduction
Advances in technology continue to impact how we seek, find, and consume information.
Many information sources have been affected by technology, including dictionaries.
Tarp (2011) explains that technology has been applied to dictionaries in varying degrees.
At the most basic level, content from dictionaries is scanned and serves as a digital copy of a
physical text. The scans are images and no real search features are available. At the next
level, technology is used to a greater extent, for example, the content might be in a database
and retrieved via searching, but technology is not used in any innovative ways, basically
providing print-based dictionaries that are now made available digitally. However, the
digital medium brings such new and different possibilities to dictionaries, it is suggested
that e-dictionaries be taken to a higher level and “designed from scratch with computer
capabilities and computer search mechanisms in mind”(Nesi, 2000, p. 140).
There are many technologies that can be employed to enhance e-dictionaries: better
search options (e.g. De Schryver, 2003; Lew, 2012; Verlinde and Peeters, 2012); more data in
the form of example sentences (De Schryver, 2003), multimedia (Lew, 2012) or linking to data
external to the dictionary (Heid et al., 2012); other innovative technologies such as filtering,
annotations or recommendations (Bothma, 2011). Technology might also be used to create
e-dictionaries customised according to the needs of users (e.g. Bothma, 2011; De Schryver,
2003; Gouws, 2014; Tarp, 2011). This could be customisation for specific situations or for the
needs and characteristics of individuals (Tarp, 2009, p. 292). The function theory of
lexicography formalises this idea that a dictionary should be customised according to
different information need situations in which a user might find him-/herself. For a short
discussion of the function theory, see Bothma and Tarp (2012). One example of this theory
must suffice here: if a person is writing a text (s)he will need different information to a
person who is doing an in-depth study of the origin of a word or phrase. As such,
the function theory suggests that an e-dictionary should be designed in such a way
(using modern technology) that only the relevant information for a specific situation
Library Hi Tech
Vol. 36 No. 2, 2018
pp. 319-338
© Emerald PublishingLimited
0737-8831
DOI 10.1108/LHT-07-2017-0144
Received 26 July 2017
Revised 16 January 2018
Accepted 23 January 2018
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0737-8831.htm
319
Heuristic
evaluation
should be given. The other irrelevant information should be withheld so that the user is
not overwhelmed.
If dictionariesare designed to make full use of the strengthsinherent in the digital medium
and break away from the traditions that have been established in their paper counterparts,
very differentinformation tools to what users are used to couldbe designed. If this is the case,
it will be important to determine if these products are usable and acceptable to users or, in
other words, the usability of e-dictionaries should be evaluated. Tullis and Albert(2008, p. 7)
confirm that the morecomplex a product becomes, the more importantit is to make sure that
the usability is good (Tullis and Albert, 2008, p. 7). The concept of usability has become
increasingly important in software design (Preece et al., 2011, p. 2). Informally it is accepted
that usabilityindicates that something canbe used for its intended purpose (Rubinet al., 2008,
p. 4). Usabilityis formally expressed in the ISO standard as “the extentto which a product can
be used by specified users to achieve specified goals with effectiveness, efficiency, and
satisfaction in a specified context of use”(ISO 9241-11, 1998), where effectiveness indicates
whether the product can be used for its intended purpose and how well it accomplishes that
particular purpose (Preece et al., 2011,p. 19), efficiency refers to the ease with which a person
can do that task or the effort thathas to be exerted in reaching the intended goal (Heid, 2011,
p. 291; Tullis and Albert, 2008, p. 8) and satisfaction refers to a person’s impression of or
attitude towards the product (Heid, 2011, p. 292; Preece et al., 2011, p. 13).
As the usability of a product has become so important, it is important that the usability
of a product or system be determined through an evaluation. Evaluation is the activity
where information about how users use a product is gathered and a judgement can be made
as to whether it is actually usable and acceptable to users (Preece et al., 2011, p. 433). There
are various methods according to which a product can be evaluated. Fernandez et al. (2011,
p. 790) explain that a usability evaluation method is “a procedure which is composed of a set
of well-defined activities for collecting usage data related to end-user interaction with a
software product and/or how the specific properties of this software product contribute to
achieving a certain degree of usability”. Examples of such methods are usability testing,
inspection methods, inquiry methods, analytical modelling or simulations.
2. Methodology
The usability of five e-dictionaries was evaluated through heuristic evaluation. Heuristic
evaluation is a type of inspection method that relies on experts to evaluate a product
according to a set of criteria (Shneiderman and Plaisant, 2010, pp. 134-136). The criteria
are used so that the evaluation is done systematically and specific usability problems can be
identified. Criteria for the evaluation ofwebpages is available, but criteria specifically for the
evaluation of e-dictionaries has been developed by Ball (2016) and Ball and Bothma (2018).
Preece et al. (2011,p. 507) confirm that general heuristics might not be specificenough for the
evaluation of certain products and evaluators can adjust the heuristics or create their own.
The expert (evaluator) typically goes through the system once to get accustomed to the
system and then doesa detailed evaluation using the specific heuristics (Barnum, 2010, p. 63;
Nielsen, 1993, pp. 158-159).
In this, study one expert reviewer did an in-depth evaluation according to the criteria
specifically designed for the evaluation of e-dictionaries by Ball (2016) and Ball and Bothma
(2018). An understanding of heuristic evaluation and how concepts such as information
architecture, advanced searching and navigation are considered during an evaluation would
also enable publishers or librarians to make informed decisions when evaluating
e-dictionaries. The finer details of each criterion are made clear through the evaluations on
the five e-dictionaries. For the sake of completeness, a summary of these criteria is included
as an Appendix.
The following five e-dictionaries were evaluated.
320
LHT
36,2
To continue reading
Request your trial