Evaluating citation visualization and exploration methods for supporting academic search tasks
| Pages | 229-254 |
| DOI | https://doi.org/10.1108/OIR-10-2014-0259 |
| Date | 13 April 2015 |
| Published date | 13 April 2015 |
| Author | Orland Hoeber,Taraneh Khazaei |
| Subject Matter | Library & information science,Information behaviour & retrieval |
Evaluating citation visualization
and exploration methods
for supporting academic
search tasks
Orland Hoeber
Department of Computer Science, University of Regina, Regina, Canada, and
Taraneh Khazaei
Department of Computer Science, Memorial University of Newfoundland,
St John’s, Canada
Abstract
Purpose –Conducting academic searches within online digital libraries can be a difficult task due to
the complexity of the searcher’s information need. The interfaces for such digital libraries commonly
use simple search features that provide limited support for the fundamental strategies that academic
searchers employ. The authors have developed a novel visualisation interface called Bow Tie
Academic Search to address some of these shortcomings, and present in this paper the findings from a
user evaluation. The paper aims to discuss these issues.
Design/methodology/approach –A controlled laboratory study was conducted to compare a
traditional search interface to Bow Tie Academic Search. In total, 24 graduate students were recruited
to perform academic searches using the two candidate interfaces, guided by specific sub-tasks that
focus on citation and keyword analysis strategies.
Findings –Although the use of the core visualisation and exploration features did not reveal
differences in retrieval effectiveness or efficiency, the query refinement features were found to be
effective. Strongly positive impressions of usefulness and ease of use of all aspects of the system were
reported, along with a preference for using Bow Tie Academic Search for academic search tasks.
Originality/value –This study provides insight into the potential value for providing visual and
interactive interfaces for supporting academic search tasks and strategies. While the quantitative
improvements over the traditional search interface were minimal, the qualitative measures illustrate
the value of Bow Tie Academic Search.
Keywords Information retrieval, Search interface, Digital library, User evaluation
Paper type Research paper
Introduction
A long-documented issue with searching within traditional libraries is the inability for
searchers to articulate their information-seeking goals (Katz, 1978). This problem may
be a result of the incomplete knowledge that inspired them to search in the first place,
or their inability to express what it is they are seeking due to a lack of terminology
(Furnas et al., 1987). The active engagement of the reference librarian within a
traditional library can overcome these problems. Within a digital library setting it is
the search interface that must take the role of the reference librarian, helping the
searcher to craft and reformulate their queries and evaluate and explore among the
search results. Online Information Review
Vol. 39 No. 2, 2015
pp. 229-254
©Emerald Group Publis hing Limited
1468-4527
DOI 10.1108/OIR-10-2014-0259
Received 29 October 2014
Second revision approved
30 January 2015
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/1468-4527.htm
This research has been made possible through the first author’s Discovery Grant, provided by
the Natural Science and Engineering Research Council of Canada.
229
Citation
visualization
and exploration
methods
The search interfaces provided by most academic-focused digital libraries follow the
common approach of providing a simple text box in which the searcher is to type their
query, and a list-based representation of the matching search results, ranked in
relevance order as determined by the search engine. Such an interface provides little
support to help the searcher articulate what it is they are seeking, or to help them find
highly relevant documents. Searchers must make extensive use of their cogn itive
abilities to read and evaluate the search results one by one, to compare multiple
documents and to consider whether they need to refine their query to focus the search
results on a more specific aspect of the topic. As a result it is common for academic
searchers to view a rather small number of papers (Nicholas et al., 2006). While this may
be partially attributed to searchers targeting specific papers, few searchers become
truly engaged in the search activity. Frustration and fatigue may prevent them from
finding the most relevant documents if they are buried deep in the ranked list or from
refining their query to more accurately describe their information-seeking goals.
Conducting an academic search is often motivated by incomplete knowledge
about a topic and a desire to learn more about it. Not only are the textual contents
of the documents important, so are the rich metadata provided by digital libraries
(e.g. authors, keywords, publication venues, publication years, citation information).
Academic searchers often employ a wide range of search strategies, depending on the
characteristics of the search task (Bates, 1989; Du and Evans, 2011; Ellis, 1989).
Without adequate interface support they must manually initiate such strategies to
explore and discover relevant documents among the search results.
Our work in the design and development of Bow Tie Academic Search (Khazaei and
Hoeber, 2012) focused on providing interactive visual representations to support
exploration among citation and keyword information. Visual encodings of citation
metadata are added to the search results list, allowing searchers to visually scan,
evaluate and compare search result items based on their citation characteristics. When
a particular document is selected, searchers are provided with an interactive
exploration interface to navigate among the forward citations (other works that
reference this document) and backward citations (references from this document to
other works). Interactive query refinement is supported via a visual overview of the
keywords associated with the top search results. Together these representations
provide concise and compact visual summaries of this metadata, and interactive
features that support the academic search strategies of citation and keyword analysis.
This paper focuses on the evaluation of Bow Tie Academic Search vs a traditional
list-based interface, with the goal of understanding the benefits of providing visual and
interactive support for academic search tasks. The remainder is organised as follows:
an overview of the related work is provided in the next section, followed by a brief
explanation of the key features of Bow Tie Academic Search. The design and results of
the controlled user study that compared this approach to a traditional list-based
interface are then provided. The paper concludes with an explanation of the primary
research contributions and an overview of future research activities.
Related work
The search interface is considered by many to be a fundamentally important aspect
of the search process (Hearst, 2009; Hoeber, 2012; White and Roth, 2009; Belkin
et al., 2011). While some approaches have sought to improve search interfaces by
automatically detecting the type of search being conducted (Awadallah et al., 2014)
230
OIR
39,2
To continue reading
Request your trial