Comparing Google to a digital reference service for answering factual and topical requests by keyword and question queries
| DOI | https://doi.org/10.1108/14684521111193201 |
| Date | 29 November 2011 |
| Published date | 29 November 2011 |
| Pages | 928-941 |
| Author | Pertti Vakkari |
| Subject Matter | Information & knowledge management,Library & information science |
Comparing Google to a digital
reference service for answering
factual and topical requests by
keyword and question queries
Pertti Vakkari
Department of Information Studies and Interactive Media,
University of Tampere, Tampere, Finland
Abstract
Purpose – This paper seeks to evaluate to what extent Google retrieved correct answers to queries
inferred from factual and topical requests in a digital Ask-a-Librarian service.
Design/methodology/approach – In total, 100 factual and 100 topical questions were picked from
a digital reference service run by public libraries. The inferred queries simulated average web queries.
They were expressed as separate keywords and as questions. The top ten retrieval results were
observed for each answer. The inspection was stopped when the first correct answer was identified.
Findings – Google retrieved correct answers to 42 percent of the topical questions and 29 percent of
factual questions by keyword queries. The performance of queries in question form was considerably
weaker. The results concerning the characteristics of queries and retrieval effectiveness are also
presented. Evaluations indicate that the public library reference services answered at least 55 percent
of the questions correctly. Thus Google did not outperform the Ask-a-Librarian service.
Originality/value – The study introduces a new way of evaluating search engines by comparing
their performance with other related services such as an Ask-a-Librarian service.
Keywords Evaluation,Search engines, Digital reference service,Effectiveness, Information transfer,
Librarians,Digital libraries, Information services
Paper type Research paper
Introduction
Search engines have enhanced the accessibility of information by providing a way to
search, identify, locate and download it. Easy access to information resources on the
internet from one’s own desk is challenging the functions of libraries, in particular public
libraries (D’Elia et al., 2002). Material and services provided by libraries are also available
on the internet. This may result in a potential change in consumer demand for the types of
services provided by libraries. People can compare the suitability and performance of the
services of the library and the internet and make their choice based on that evaluation.
It is typical to evaluate search engines by comparing them with each other
(Lewandowski, 2008). Similarly digital Ask-an-Expert-services – including librarians
– are evaluated by comparing thes e services with each other (Janes et al., 2001). It is
rare to evaluate search services by comparing them with different types, e.g. how
search engines function compared to digital library services such as question
answering or even retrieving information from a digital library (see Harper et al., 2008;
Shachaf, 2009; Vakkari and Taneli, 2009). However it is necessary to evaluate search
engines with other appropriate services such as those provided by libraries, because
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1468-4527.htm
OIR
35,6
928
Received 15 September 2010
Accepted 27 March 2011
Online Information Review
Vol. 35 No. 6, 2011
pp. 928-941
qEmerald Group Publishing Limited
1468-4527
DOI 10.1108/14684521111193201
To continue reading
Request your trial