The history and development of transaction log analysis
| DOI | https://doi.org/10.1108/eb047884 |
| Pages | 41-66 |
| Published date | 01 February 1993 |
| Date | 01 February 1993 |
| Author | Thomas A. Peters |
| Subject Matter | Information & knowledge management,Library & information science |
THE HISTORY AND DEVELOPMENT OF
TRANSACTION LOG ANALYSIS
Thomas A. Peters
CONTENTS
Introduction
Reported Research
Actual Use of Operational IR Systems
Studies of Actual Public Use
Commands, Response Time, Session Lengths
Chains of Commands and State to State Transitions
Inter-Mode Analyses—Menu and Command
Intersite and Interinstitutional Analyses: Use of
Union Scoping
Intersystem Analyses
Errors, Zero-Hits, Missed Opportunities, Failures,
and Their Causes
Online Help and Other Instructional Opportunities
Analyses of Specific Search States
Extent of Match Studies
Other Access Points and Advanced Features
Printing and Downloading Behavior
User Persistence
Quitting Behavior and Situations
Analyses of Special Public User Groups
Longitudinal Studies and Variations over Time
Replications of Previous TLA Research
TLA Combined with Other Methods
Analyses of Other Databases
Analyses of Environmental Factors
Studies of Actual Staff Use
Studies of Experimental Use of IR Systems
Discussions of TLA as a Method
Other and Future Uses of TLA
Conclusion
Sidebar 1: Transaction Log Analyses in Progress
Sidebar 2: Transaction Log Analyses Sorted by System
(1980 to Present)
INTRODUCTION
The purpose of
this
article is to present an over-
view of
the
history and development of transaction log
analysis (TLA) in library and information science
research. Organizing a literature review of the first
twenty-five years of TLA poses some challenges and
requires some decisions. The primary organizing
principle could be a strict chronology of the published
research, the research questions addressed, the automat-
ed information retrieval (IR) systems that generated
the data, the results gained, or even the researchers
themselves. The
group
of active transaction
log
analyz-
ers remains fairly small in number, and researchers
who use transaction logs tend to use this method more
than once, so tracing the development and refinement
of
individuals'
uses of the methodology could provide
insight
into the
progress of the method
as a
whole.
For
example, if
we
examine how researchers like
W.
David
Penniman, John
Tolle,
Christine
Borgman,
Ray
Larson,
and Micheline Hancock-Beaulieu have modified their
own
understandings and applications of the method over
time,
we may
get an accurate sense of
the
development
of all applications.
Another challenge involves not imposing an
artificial order on a method whose applications have
not always been conceived with a clear sense of a
developing methodology. The first quarter century of
TLA is not particularly noteworthy for its self-reflec-
tiveness and cumulating knowledge
base.
Often transac-
tion log
analyzers are intent, perhaps rightly
so,
on the
object of
analysis,
rather than on the new method they
are employing.
This article is an organization of the literature of
transaction
log analysis
that
reveals
the diversity of the
types of projects that have employed the method. The
primary purpose is to review the history and develop-
Peters is coordinator of collection development,
Mankato State University.
—
HISTORY
AND
DEVELOPMENT
—
ISSUE 42
—
11:2 (1993) 41
SIDEBAR
1:
TRANSACTION
LOG
ANALYSES
IN
PROGRESS
In early
1993 the
following transaction
log
analyses were known
by
the author
to
be in progress.
Although certainly
not
exhaustive,
the
list indicates
some current applications of the methodology. Terry
Ballard at Adelphi University
is
using transaction logs
to study how members of the library staff search
the
Innopac online catalog. Searches conducted
at
selected
staff terminals
are
being compared
to
searches
performed
at
selected in-library public access termi-
nals.
Nirmala Bangalore
at
the University of Illinois
at Chicago
is
using data collected
in
March 1992
to
replicate the research conducted by Arlene Taylor
in
the early 1980s. Martin Kurth
and Tom
Peters
are
using transaction
log
data from
the
WLN-based
system
at the
University of Missouri
at
Kansas City
to study
the
online browsing behavior
of
subject
searchers as they interact with an alphabetized list of
subject headings. Beth Sandore
and
Kathy Ryan
at
the University
of
Illinois
at
Urbana-Champaign
are
using transaction
log
analysis together with protocol
analysis to
examine keyword uncontrolled vocabulary
subject searching in journal index databases (Wilson,
ERIC,
and
Current Contents) accessible through
the
BRS search software and
the
local online
catalog.
For
his doctoral research, David Weisbrod
at
Rutgers
University is using transaction log and interview data
(originally used by Belkin,
Chang,
Downs,
Saracevic,
and Zhao)
to
investigate
the
potential effectiveness
of an end-user thesaurus within an online catalog.
He
is measuring
and
evaluating
the
effectiveness
of
the
subject searcher
in
gaining initial access to the online
catalog database.
The
problems
of
failed subject
searches
and
abandonment
of
searches following
no
hits
are being
examined.
An end-user thesaurus would
address
the
variable
use of
human language
by
providing a semantic knowledge base without enhanc-
ing individual bibliographic
records.
Stephen Wiber-
ley,
Jr.,
Robert Daugherty, and James Danowski
are
continuing their research into user persistence
in
displaying online catalog
postings.
They are using data
collected via
the
transaction logging facility of
LUIS,
the NOTIS-based online catalog
at
the University
of
Illinois at Chicago. They have replicated the searches
to gather additional data.
A
post-search questionnaire
also
was
administered. These current projects
are
indicative
of the
diverse questions being explored
through transaction
log
analysis.
ment of TLA, not
to
compare, contrast,
or
summarize
the findings
of
previous
transaction log research.
The
results of transaction
log
analyses,
even those purport-
edly pursuing the same research
question,
are amazing-
ly diverse.
An
analysis
of
the diversity
of
the results
would involve
a
lengthy investigation.
In his
review
of major transaction
log
analyses
of
online catalogs,
Simpson chose
to
organize
the
review
by
type
of
analysis,
rather
than
chronologically.1
This
review will
adhere
to
that principle, organizing
the
published
research according
to
the distinguishing
qualities
of the
method (e.g.,
the
facet
of the
search session under
scrutiny,
the
user group,
the
type
of
information
system,
and so on),
rather than
by the
results. Each
section, in turn, will adhere loosely to a chronological
arrangement
so
that, within each section
at
least, some
sense of the historical development of the method may
emerge.
Several previous literature reviews including
transaction
log analyses have
appeared,
each
providing
a different perspective
on the use of
this method
for
data collection and analysis.2 The literature review by
Borgman and Rice
was
unique in that
they
covered the
use of TLA relating
to
both IR systems and computer-
ized communication systems. Borgman's extensive
literature review of research
into
use of IR systems was
notable
in
this context because
she
called
for
more
research into
the use of
online catalogs
by
librarians
and other skilled users, rather than
by
typical public
users.
Simpson limited
his
review just
to
online catalog
use
and
presented
an
overview
of
approximately
two
dozen important studies. During his thorough review
of subject access
to
information contained
in
online
catalogs, Larson discussed
the
subject searching
frequencies measured via transaction
log
analysis.
The
recent annotated bibliography published
by
Peters,
Kaske,
and
Kurth
also
functioned
as a type
of literature
review.3
Transaction
log
analysis
can be
conceptualized
both
as a
form
of
system monitoring
and as a
way
of
observing,
usually unobtrusively, human behavior. As
a historical phenomenon, however, transaction
log
analysis evolved
out of the
desire
to
monitor
the
performance of computerized IR
systems.
Even though
psychologists like Parker
and
Paisley quickly
saw in
computerized IR systems new opportunities
for
studying
human behavior, system designers and managers first
developed
and
utilized transaction logging facilities.4
A general movement
in the
developing historical
understanding of transaction log analysis has been from
the system monitoring end
of
the
spectrum toward
the
observation of human behavior
in a
specific, computer-
ized environment.
The
technique of generating and analyzing transac-
tion
logs appears to
be approximately twenty-five years
old. One of the earliest, if not the
first,
research project
utilizing transaction
log
analysis
was
conducted
by
Meister
and
Sullivan
in
1967.5
If
the development
of
transaction log analysis
can be
divided
into
phases,
the
first phase (from
the
mid-1960s
to the
late 1970s)
placed emphasis
on
evaluating system performance,
rather than
on
user behavior
and
performance.
The
42 LIBRARY HI
TECH
— THOMAS A. PETERS
—
To continue reading
Request your trial