Assessing metasearch engine performance
| DOI | https://doi.org/10.1108/14684520911011007 |
| Pages | 1058-1065 |
| Date | 27 November 2009 |
| Published date | 27 November 2009 |
| Author | Hamid Sadeghi |
| Subject Matter | Information & knowledge management,Library & information science |
Assessing metasearch engine
performance
Hamid Sadeghi
Department of Computer Engineering, Islamic Azad University,
Hashtgerd, Iran
Abstract
Purpose – The purpose of this paper is to present a new method for evaluating the performance of
metasearch engines (MSEs), which was used in the reported study to investigate which of eight
popular MSEs (Clusty, Dogpile, Excite, Mamma, MetaCrawler, Search.com, WebCrawler and
Webfetch) is the best.
Design/methodology/approach – This research evaluated the performance of eight MSEs. For
each MSE the average of closeness degrees between its ranked result list and those of its underlying
search engines (SEs) was measured. Next, these measures were compared to each other to determine
which MSE gives the best performance. Furthermore the experiment was repeated ten times with ten
different queries to reach a stable result.
Findings – The findings revealedthat Dogpile outperformed all the others, followed by MetaCrawler,
Excite, Webfetch and then Mamma. MetaCrawler and WebCrawler had almost the same performance
and occupied the next positions. Clusty and Search.com performed poorly in comparison to the others.
Practical implications – The findings of this research would be useful for MSE designers as well as
helping the numerous users of MSEs to choose a truly effective one.
Originality/value – This paper provides anovel method for assessing the performance of MSEs and
valuable experimental results on eight popular ones.
Keywords Performance appraisal, Search engines,Information retrieval, Worldwide web
Paper type Research paper
Introduction
Searching is a crucial activity on the web (Madden, 2003; Fallows, 2004) and search
engines (SEs) are the powerful search tools for locating information in this
environment. However, it is impossible for any single SE to index the entire web and
large SEs cover only a fraction of it (Bharat and Broder, 1998; Lawrence and Giles,
1999). Recently, Spink et al. (2006) conducted a large-scale study across the four most
popular web SEs (MSN, Google, Yahoo and Ask Jeeves) to measure the overlap of
search results on the first results page. Their findings showed that the percentage of
total results unique to only one of the four web SEs was 84.9 per cent, shared by two of
the three web SEs was 11.4 per cent, shared by three of the web SEs was 2.6 per cent,
and shared by all four web SEs was 1.1 per cent. This small degree of overlap shows a
single SE cannot be utilised effectively for finding information on the web.
By combining the coverage of multiple SEs through a system called a metasearch
engine (MSE), a much higher percentage of the web can be searched. An MSE is a
system that provides unified access to multiple existing SEs (Meng et al., 2002). When
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1468-4527.htm
The author would like to thank the Islamic Azad University, Hashtgerd Branch for the financial
support of this research.
OIR
33,6
1058
Refereed article received
16 November 2008
Approved for publication
17 June 2009
Online Information Review
Vol. 33 No. 6, 2009
pp. 1058-1065
qEmerald Group Publishing Limited
1468-4527
DOI 10.1108/14684520911011007
To continue reading
Request your trial