Meeting Big Data challenges with visual analytics. The role of records management
| DOI | https://doi.org/10.1108/RMJ-01-2014-0009 |
| Pages | 122-141 |
| Date | 15 July 2014 |
| Published date | 15 July 2014 |
| Author | Victoria Louise Lemieux,Brianna Gormly,Lyse Rowledge |
| Subject Matter | Information & knowledge management,Information management & governance |
Meeting Big Data challenges
with visual analytics
The role of records management
Victoria Louise Lemieux, Brianna Gormly and Lyse Rowledge
School of Library, Archival and Information Studies,
The University of British Columbia, Vancouver, Canada
Abstract
Purpose – This paper aims to explore the role of records management in supporting the effective use
of information visualisation and visual analytics (VA) to meet the challenges associated with the
analysis of Big Data.
Design/methodology/approach – This exploratory research entailed conducting and analysing
interviews with a convenience sample of visual analysts and VA tool developers, afliated with a major
VA institute, to gain a deeper understanding of data-related issues that constrain or prevent effective
visual analysis of large data sets or the use of VA tools, and analysing key emergent themes related to
data challenges to map them to records management controls that may be used to address them.
Findings – The authors identify key data-related issues that constrain or prevent effective visual
analysis of large data sets or the use of VA tools, and identify records management controls that may be
used to address these data-related issues.
Originality/value – This paper discusses a relatively new eld, VA, which has emerged in response
to meeting the challenge of analysing big, open data. It contributes a small exploratory research study
aimed at helping records professionals understand the data challenges faced by visual analysts and, by
extension, data scientists for the analysis of large and heterogeneous data sets. It further aims to help
records professionals identify how records management controls may be used to address data issues in
the context of VA.
Keywords Big Data, Records management, Information governance, Information visualisation,
Visual analytics
Paper type Research paper
Introduction
Data are generated and collected in unprecedented volumes today, giving rise to the
notion of “Big Data”. What is Big Data? The answer is difcult to pinpoint. As the name
suggests, one dening characteristic of Big Data is its size. At what volume data become
big remains an open question, however, with some suggesting that it comprises data at
the scale of exabytes, while others argue for zettabytes or yottabytes (Heer and Kandel,
2012). A more formal denition of the term suggests that it is data “with sizes beyond the
ability of commonly used software tools to capture, curate, manage, and process the data
within a tolerable elapsed time” (Snijders et al., 2012). Other denitions emphasise not
just the increasing volume or amount of data, but also its velocity (speed of data in and
out), and variety (range of data types and sources) (Gartner, 2011). Some writers also
include veracity (the biases, noise and abnormality in data) as an additional dening
characteristic (Normandeau, 2013). On the difculties of dening Big Data, one
participant in this study wryly commented, “I don’t know if there’s any clear denition
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0956-5698.htm
RMJ
24,2
122
Received 20 January 2014
Revised 19 May 2014
Accepted 3 June 2014
Records Management Journal
Vol. 24 No. 2, 2014
pp. 122-141
© Emerald Group Publishing Limited
0956-5698
DOI 10.1108/RMJ-01-2014-0009
of what Big Data is, but it’s kind of like big enough to become annoying I guess.” (S4,
page 9 lines 23-24).
Regardless of precise denition, exponential growth, speed and variety in data have
created new challenges not only in the management of vast amounts of heterogeneous
data, but also in how to make sense of it all. Within organisations, a growing cadre of
workers, with titles such as “business analyst”, “data analyst” and “data scientist”
(Kandel et al., 2011), are harnessing new tools, practices and solutions. Among these
tools is visual analytics (VA), often dened as “the science of analytical reasoning
facilitated by interactive visual interfaces” (Thomas and Cook, 2005). As powerful and
promising a tool as VA is, difculties with identifying and “wrangling” the data needed
for visual analysis are proving to be major barriers to its adoption and effectiveness.
This paper presents the results of an exploratory study aimed at understanding the data
issues related to VA and ways in which records management controls might be applied
to address these issues[1].
Information visualisation, VA and the challenge of analysing Big Data
As the volume, velocity and variety of data available to business people, scientists and
the public increase, effective use of data becomes more challenging. Standard tools for
data analysis and exploration fall short as a means of keeping up to date with the ood
of data. Take, for example, a simple statistical query, such as determining the
characteristics of a population from the data presented in Table I.
The human information processing system simply cannot hold information in working
memory long enough to extract relevant patterns from the data. Using visualisation lightens
this burden, however, because encoding information visually relieves the demand on our
memories and allows patterns to “pop out” (Treisman, 1985) and grab our attention
Table I.
Financial data showing
the performance of
different products. How
easy is it to spot the areas
of the business with
negative prot?
Summary of nancial performance
Product type
Central East South West
Total
sales
Total
prot
Total
sales
Total
prot
Total
sales
Total
prot
Total
sales
Total
prot
Coffee
Amaretto $14,011 $5,105 $2,993 $1,009 $9,265 ⫺$1,225
Columbian $28,913 $8,528 $47,305 $27,253 $21,004 $8,707 $30,337 $11,253
Decaf Irish $25,155 $9,632 $6,261 $2,727 $11,392 $2,933 $18,235 ⫺$1,305
Espresso
Caffe latte $15,442 $3,872 $20,458 $7,502
Caffe mocha $35,218 $14,649 $16,645 ⫺$6,230 $14,163 $5,201 $18,876 $4,064
Decaf espresso $24,495 $8,869 $7,722 $2,410 $15,384 $5,030 $30,578 $12,302
Reg. espresso $24,036 $10,062
Herbal tea
Chamomile $35,570 $14,434 $2,194 $765 $11,186 $3,160.00 $25,632 $6,852
Lemon $21,978 $6,251 $27,175 $7,901 $14,497 $2,593.00 $32,274 $13,120
Mint $9,337 $4,069 $11,992 ⫺$2,242 $14,380 $4,330
Darjeeling $30,289 $10,772 $14,095 $6,497 $28,759 $11,780
Earl Grey $32,881 $10,331 $6,505 $3,405 $27,387 $10,425
Green tea $5,211 $1,227 $11,571 $5,654 $16,063 ⫺$7,109
123
Meeting Big
Data challenges
with visual
analytics
To continue reading
Request your trial