Users’ relevance criteria for video in leisure contexts
| Date | 08 January 2018 |
| DOI | https://doi.org/10.1108/JD-06-2017-0081 |
| Pages | 62-79 |
| Published date | 08 January 2018 |
| Author | Sarah Ahmed A. Albassam,Ian Ruthven |
| Subject Matter | Library & information science,Records management & preservation,Document management,Classification & cataloguing,Information behaviour & retrieval,Collection building & management,Scholarly communications/publishing,Information & knowledge management,Information management & governance,Information management,Information & communications technology,Internet |
Users’relevance criteria for video
in leisure contexts
Sarah Ahmed A. Albassam and Ian Ruthven
Department of Computer and Information Sciences, University of Strathclyde,
Glasgow, UK
Abstract
Purpose –The purpose of this paper is to understand how typical users of YouTube judge the relevance
of videos in leisure contexts; what are the reasons users give when judging video material as relevant or
not relevant?
Design/methodology/approach –A naturalistic diary was performed in which 30 participants completed
diaries providing details on their video relevance criteria. The analysis revealed 28 relevance criteria grouped
into eight categories.
Findings –In total, 28 relevance criteria were identified through the analyses of the diaries’content and they
were grouped into eight categories. The findings revealed that criteria related to the content of the video are
the most dominant group of criteria with topicality being the most dominant criterion. There is a considerable
overlap between leisure relevance criteria and previous relevance criteria studies, but the importance of these
criteria varies among different contexts. New criteria, e.g. habit emerged from the data which tend to be more
related to leisure contexts.
Research limitations/implications –The decision to follow a naturalistic approach reduced the level of
control on the study. A further limitation can be found in the participants’sample used in this study, all the
participants of the main study were university or college students.
Practical implications –This study attempted t o enrich the current lit erature by investiga ting users’
video relevance crite ria in leisure context s. This investigatio n might have implicatio ns on the design of
video search systems.
Originality/value –Previous relevance criteria studies focussed on work contexts and the information
judged was mainly in text format. This paper outlines new insights by investigating video relevance criteria
in leisure context.
Keywords User studies, Casual leisure, Naturalistic diary study, Relevance criteria, User judgement,
Video retrieval
Paper type Research paper
1. Introduction
Relevance is a core concept in information retrieval (IR). The ultimate goal of all IR systems
is to retrieve relevant documents that satisfy user information needs. Many studies have
attempted to understand how people judge the relevancy of retrieved documents resulting
in an enormous amount of work concerning user relevance judgement, behaviour and
factors affecting their relevance decisions. In the area of user relevance, researchers are
especially interested in what reasons users give to judge documents as relevant or not
relevant. In other words, what are the relevance criteria that users apply when making a
relevance judgement decision? Although there is a rich literature on different aspects
of relevance criteria, there are still open questions. For example, there is a lack of knowledge
regarding how users apply relevance criteria in video relevance judgements. A few studies
have focussed on video relevance, such as Yang (2005), however, these studies are limited to
work-related tasks.
The affordability of technology and the internet facilitate the use of IR systems not just
for work-related tasks but also during leisure time (Elsweiler et al., 2011; Hartel, 2003).
Journal of Documentation
Vol. 74 No. 1, 2018
pp. 62-79
© Emerald PublishingLimited
0022-0418
DOI 10.1108/JD-06-2017-0081
Received 2 June 2017
Revised 29 August 2017
Accepted 31 August 2017
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0022-0418.htm
This work was supported by the King Saud University, Saudi Arabia. The authors wish to thank
the participants who participate in the study, as well as the anonymous reviewers for their
constructive comments.
62
JD
74,1
Little, though is known about the relevance criteria that are important to users in leisure
contexts and how these might be different from other contexts. Thus, we need research to
fill in the gap of knowledge regarding video relevance criteria in leisure context.
The importance of this study lies in its attempt to enrich the current literature by
investigating users’video relevance criteria in leisure contexts. By shedding light on this
uncovered area of research, the IR community will gain a deeper understanding of how
users make their relevance judgement decisions in a leisure context. Furthermore, the
findings of this study will have implications on the design of IR systems. Retrieval systems
designed for users with leisure needs might be different from those targeting users with
work-related needs. In addition, the design of video retrieval systems may also be different
from text retrieval systems. The comparisons of relevance criteria mentioned among these
different media and situations will guide designers of different IR systems. As leisure users
have different needs and motivations, investigating relevance criteria in leisure contexts will
help in designing systems that address their needs.
This study was conducted to investigate the relevance criteria users apply when judging
videos in a leisure context and to what extent do these criteria match the criteria mentioned
in the previous literature of text retrieval and/or work task contexts?
2. Related work
2.1 Relevance criteria studies
Early research has significantly contributed to the user relevance criteria literature and has
identified key criteria and synthesized them into different categories. Schamber (1991)
mentioned 22 criteria classified into ten categories gathered from 30 occupational users of
weather information. In a similar work, Barry (1994) identified 23 criteria grouped into seven
categories from faculty and student users. Later, Barry and Schamber (1998) combined and
compared the results of their previous studies. The comparison yielded ten criteria in
common. The criteria are: depth/scope/specificity, accuracy/validity, clarity, currency,
tangibility, quality of sour ces, accessibility, avai lability of information/so urces of
information based on their research, verification and affectiveness.
Based on the work of Barry and Schamber (1998), other studies emerged to cover
different aspects of relevance criteria. For example, group of studies in relevance criteria
literature (Wen et al., 2006; Tombros et al., 2005; Balatsoukas and Ruthven, 2012) have
focussed on the factors that guide the relevance criteria selection process. Topic familiarity,
searching task type and the ranking order of the result list are some of the factors that found
to have an effect on the users’relevance criteria choices.
The findings of relevance criteria studies revealed a list of relevance criteria in text
retrieval systems and encouraged researchers to examine whether the same criteria are
applied in non-text retrieval systems. Studies have emerged to investigate user relevance
criteria in different media, e.g., image retrieval, music retrieval and video retrieval.
Laplante (2010) explored relevance judgements in the context of music retrieval for leisure
purposes. The findings indicated that in the context of music retrieval, criteria related to the
participant’s beliefs and preferences have a significant influence on relevance judgement
decision and topicality was not mentioned by the study participants. Choi and Rasmussen
(2002) attemptedto discover the relevance criteria that users employ when making relevance
judgements of images. They reported that topicality dominates among other criteria.
In addition, other criteria like image quality and clarity seems to be important to users.
They also noticeddifferences in the priority of applying the criteria atdifferent search stages
with the importanceof criteria such as appeal of information and accessibilityincreased as the
search process progress whilst the importance of topicality decreases.
There were also limited studies that investigated relevance criteria in video retrieval.
In her PhD study, Yang (2005) explored the criteria people applied when making video
63
Users’
relevance
criteria
To continue reading
Request your trial