A practical guide for human lab experiments in information systems research. A tutorial with Brownie
| Pages | 228-256 |
| Published date | 14 August 2017 |
| Date | 14 August 2017 |
| DOI | https://doi.org/10.1108/JSIT-06-2017-0049 |
| Author | Dominik Jung,Marc Adam,Verena Dorner,Anuja Hariharan |
| Subject Matter | Information & knowledge management,Information systems,Information & communications technology |
A practical guide for human lab
experiments in information
systems research
A tutorial with Brownie
Dominik Jung
Institut für Informationswirtschaft und Marketing,
Karlsruher Institut für Technologie, Karlsruhe, Baden-Württemberg, Germany
Marc Adam
Faculty of Engineering and Built Environment, The University of Newcastle,
Callaghan, New South Wales, Australia, and
Verena Dorner and Anuja Hariharan
Institut für Informationswirtschaft und Marketing,
Karlsruher Institut für Technologie, Karlsruhe, Baden-Württemberg, Germany
Abstract
Purpose –Human lab experimentshave become an established method in informationsystems research for
investigating user behavior,perception and even neurophysiology. The purpose of this paper is to facilitate
experimental research by providinga practical guide on how to implement and conduct lab experiments in
the freely availableexperimental platform Brownie.
Design/methodology/approach –Laying the groundwork of the tutorial, the paper first provides a
brief overview of common design considerations for lab experiments and a generic session framework.
Building on the usecase of the widely used trust game, the paper then coversthe different stages involved in
running an experimentalsession and maps the conceptual elements of the study design to the implementation
of the experimentalsoftware.
Findings –The paper generates findings on how computerized lab experiments can be designed and
implemented. Furthermore, it maps out the design considerations an experimenter may take into account
when implementing an experiment andorganizing it along a session structure (e.g. participant instructions,
individualand group interaction, state and trait questionnaires).
Originality/value –The paper reduces barriers for researchersto engage in experiment implementation
and replication by providing a step-by-step tutorial for the design and implementation of human lab
experiments.
Keywords Brownie, Experimental software, Human lab experiments, Tutorial
Paper type Technical paper
1. Introduction
In information systems (IS) research, human lab experiments have become an established
method for gathering behavioral, perceptional and even neurophysiological data from users
interacting with information technology (IT) in a controlled environment. Building on the
The authors thank Ewa Lux, An Thuy Tien Luong and Raphael Rissler for their valuable feedback
on an earlier version of this paper.
JSIT
19,3/4
228
Received29 June 2017
Revised31 October 2017
Accepted2 November 2017
Journalof Systems and
InformationTechnology
Vol.19 No. 3/4, 2017
pp. 228-256
© Emerald Publishing Limited
1328-7265
DOI 10.1108/JSIT-06-2017-0049
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/1328-7265.htm
early groundwork in experimental psychology and economics (Falk and Heckman, 2009;
Hertwig and Ortmann, 2001),IS scholars use lab experiments:
in behavioral science research to test and refine theories of user behavior; and
in design science research to evaluate and improve IS artefacts in human–computer
interaction.
As the effort and technical knowledge involved in developing and conducting experiments
is substantial, researchers commonly build on existing experimental software when
carrying out their studies. Experimental software, such as z-Tree (Fischbacher, 2007),
Noldus Observer (Noldus, 1991), OpenSesame (Mathôt et al., 2012), N4U (Munir et al.,2014)
or Brownie (Hariharan et al.,2017), enables researchers to build on a foundation of
specialized libraries and interfaces (e.g., for event logging, group interaction, or stimuli
presentation),in order to build programs for a specific research investigation by lettingthem
organize the sequenceof tasks and events during an experimental session and tailorexisting
software features.
This technical paper aims at guiding and facilitating the design and implementation of
new experiments as well as the replication of previously published experimental research.
To this end, we present an experimental session framework and showcase the software
implementation of a specific experimental investigation on trusting behavior in online
interactions. Thereby, we build on the freely available experimental software platform
Brownie (Behavioral Research of individuals and groups using Web and NeuroIS
Experiments; Hariharan et al.,2017). Brownie is an open source platform for conducting
behavioral experiments in the lab, enabling scholars in IS research and cognate areas to
investigate human–computer interaction in individual as well as in group settings. The
platform was specifically designed for conducting human lab experiments with
neurophysiological measurements in IS research [e.g. recordings of electrocardiography
(ECG), electrodermal activity (EDA), electroencephalography (EEG), functional magnetic
resonance imaging (fMRI), photoplethysmograph (PPG)] in the field of NeuroIS (vom Brocke
and Liang, 2014;Riedl and Léger, 2016), but can be used for a wide range of computerized
experiments in the lab, with or without neurophysiological measurements. We decided to
use Brownie as theunderlying platform for this tutorial because:
the platform has been designed and evaluated specifically for the needs of IS
researchers;
it is Java-based and hence builds on a widely used programming language;
all components are open source and extensible; and
its architecture takes into account a number of methodological aspects (e.g.
treatment design, matching procedures) with corresponding technical
functionalities (e.g. interaction between the participants, sequence of experimental
tasks).
In addition, Browniefacilitates flexibility in storing different types of user data, with desired
granularity, and is extensible to meet researchers’technological needs (e.g. timestamp and
data handling, integrationof sensors and websites), that are a necessity for IS research.
The tutorial is organized along six stages of conducting a human lab experiment,
conceptualized in a genericexperimental session framework, providing a practicalguide for
how each of the steps can be implemented in Brownie. We illustrate our tutorial with a
frequently researched topicin IS: antecedents of trust in online transactions (Du and Huang,
2013;Riedl et al., 2014;Hawlitschek et al., 2016). It is couched along the standardized
Guide for
human lab
experiments
229
To continue reading
Request your trial