Engineering social media driven intelligent systems through crowdsourcing. Insights from a financial news summarisation system

Date08 August 2016
DOIhttps://doi.org/10.1108/JSIT-03-2016-0019
Pages255-276
Published date08 August 2016
AuthorMartin Sykora
Subject MatterInformation & knowledge management,Information systems,Information & communications technology
Engineering social media driven
intelligent systems through
crowdsourcing
Insights from a nancial news summarisation
system
Martin Sykora
School of Business and Economics, Loughborough University,
Loughborough, UK
Abstract
Purpose – The purpose of this paper is to explore implicit crowdsourcing, leveraging social media in
real-time scenarios for intelligent systems.
Design/methodology/approach A case study using an illustrative example system, which
systematically used a custom social media platform for automated nancial news analysis and
summarisation was developed, evaluated and discussed. Literature review related to crowdsourcing
and collective intelligence in intelligent systems was also conducted to provide context and to further
explore the case study.
Findings It was shown how, and that useful intelligent systems can be constructed from
appropriately engineered custom social media platforms which are integrated with intelligent
automated processes. A recent inter-rater agreement measure for evaluating quality of implicit crowd
contributions was also explored and found to be of value.
Practical implications – This paper argues that when social media platforms are closely integrated
with other automated processes into a single system, this may provide a highly worthwhile online and
real-time approach to intelligent systems through implicit crowdsourcing. Key practical issues, such as
achieving high-quality crowd contributions, challenges of efcient workows and real-time crowd
integration into intelligent systems, were discussed. Important ethical and related considerations were also
covered.
Originality/value – A contribution to existing theory was made by proposing how social media Web
platforms may benet crowdsourcing. As opposed to traditional crowdsourcing platforms, the
presented approach and example system has a set of social elements that encourages implicit
crowdsourcing. Instances of crowdsourcing with existing social media, such as Twitter, often also
called crowd piggybacking, have been used in the past; however, using an entirely custom-built social
media system for implicit crowdsourcing is relatively novel and has several advantages. Some of the
discussion in context of intelligent systems construction are novel and contribute to the existing body
of literature in this eld.
Keywords Natural language processing, Social media, Crowdsourcing, Crowd-Powered systems,
Intelligent systems
Paper type Research paper
1. Introduction
Online-based crowdsourcing has opened up new and interesting applications in areas,
where cognitive capabilities and the collective intelligence of the crowd allow for
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/1328-7265.htm
Crowdsourcing
255
Received 2 March 2016
Revised 15 May 2016
Accepted 20 May 2016
Journalof Systems and
InformationTechnology
Vol.18 No. 3, 2016
pp.255-276
©Emerald Group Publishing Limited
1328-7265
DOI 10.1108/JSIT-03-2016-0019
accurate solutions, where traditionally, individuals with expert knowledge were
required to approach the tasks at hand (Brabham, 2008). Crowdsourcing platforms now
provide for a scalable human intelligence processing resource that can be tapped into by
researchers and system engineers alike. Especially, the elds of articial intelligence
(AI) and machine learning (ML) have beneted from the readily available crowds (on
platforms such as Amazon Mechanical Turk; Paolacci et al., 2010), which can now
annotate huge and often complex data sets in a fraction of the usual time and costs
required for annotation tasks. These crowd-annotated data sets in turn are used to train
and develop better and more accurate AI/ML models. However, computer-based
systems purely relying on AI and ML have not delivered truly intelligent systems,
which is where a closer integration with human cognitive and reasoning capabilities, if
integrated effectively, hold considerable promise. To this end, Lasecki (2014), for
instance, provides several inspirational examples of crowd-driven intelligent systems.
He points out, however, the challenges of seamless integration of the crowd and
especially design for on-demand and real-time intelligent systems, where collecting and
motivating crowd contributions in real time, is a signicant challenge, which, to date,
has mostly been overlooked in academic research.
In this paper, we argue for the benets of implicit crowdsourcing by harnessing a
crowd’s collective intelligence and cognitive capabilities through social media. We
discuss how social media-based websites can be directly used within crowd-driven
intelligent systems. An example social media-based, crowd-driven intelligent system,
for news analysis is presented and evaluated and related issues and insights from its
development are discussed. Specically, issues of achieving high enough quality crowd
contributions and challenges of efcient workow and real-time crowd integration into
intelligent systems are considered. As opposed to traditional crowdsourcing platforms,
the presented system has a set of social elements that encourages implicit
crowdsourcing. Instances of crowdsourcing with existing social media, such as Twitter,
often called piggybacking, have been used in the past (Grevet and Gilbert, 2015);
however, using an entirely custom-built social media system for implicit crowdsourcing
is relatively novel and has several advantages.
The remainder of this paper is organised as follows. Section 2 introduces some
background on crowdsourcing, related intelligent systems and the approach to
crowdsourcing through social media integration into intelligent systems applications.
Section 3 presents the crowd-driven news analysis system and study, and Section 4
provides a discussion and limitations to the presented work. The paper is nally
concluded in Section 5.
2. Related literature and theoretical background
The term crowdsourcing was coined about 10 years ago, by Howe (2006), although, by
2012, Estellés-Arolas and González-Ladrón-de-Guevara (2012) reviewed over 40
different denitions of the term. Bringing together the various denitions, the main
element of their integrated denition was highlighted as:
Crowdsourcing is a type of participative online activity in which an individual, an institution,
a non-prot organization, or company proposes to a group of individuals of varying
knowledge, heterogeneity, and number, via a exible open call, the voluntary undertaking of a
task. The undertaking of the task, of variable complexity and modularity, and in which the
JSIT
18,3
256

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT