Recommendations for Reporting Sample and Measurement Information in Experience Sampling Studies
| Published date | 01 April 2022 |
| Author | Eric D. Heggestad,Liana Kreamer,Mary M. Hausfeld,Charmi Patel,Steven G. Rogelberg |
| Date | 01 April 2022 |
| DOI | http://doi.org/10.1111/1467-8551.12489 |
British Journal of Management, Vol. 33, 553–570 (2022)
DOI: 10.1111/1467-8551.12489
Recommendations for Reporting Sample and
Measurement Information in Experience
Sampling Studies
Eric D. Heggestad ,1Liana Kreamer,1Mary M. Hausfeld,1Charmi Patel2
and Steven G. Rogelberg1
1University of North Carolina at Charlotte, 9201 University City Boulevard, Charlotte, NC, 28223, USA
2Henley Business School, Whiteknights, Reading, RG6 6UD, UK
Corresponding author email: edhegges@uncc.edu
Over the last couple of decades, studies using the experience sampling methodology
(ESM) have been used with increasing frequencywithin the management-related sciences
as the method allows researchers the opportunity to investigatequestions involving ongo-
ing, dynamic, intra-individual processes. Given the longitudinal nature of the method-
ology and the resulting multi-level data structure, there are sample- and measurement-
related issues that make ESM studies different from other methods commonly used in
management research. Consequently, ESM studies have demands for reporting sample-
and measurement-related information that differ from more commonly used methods. In
the current paper, we review the conceptual foundations of sample and measurement is-
sues in ESM studies and report the ndings of a survey of the ESM studies to identify
current reporting practices. Wethen offer clear, easy to implement recommendations for
reporting sample- and measurement-related aspects of ESM studies. We hope that these
recommendations will improve reportingof ESM studies and allow readers the opportu-
nity to more fully and comprehensively evaluate the research presented.
Introduction
Many management scholars are interested in re-
lational processes, including those between spe-
cic organizational members (e.g. employees) and
those between individual employees and the orga-
nization itself. These relational processes are, by
their very nature,dynamic. For instance,how a fol-
lower feels about their leader is likely to change
based on the actions of the leader, perhaps even
on a day-to-daybasis. Likewise,an employee’s level
of organizational commitment may change based
on the issuance of a new policy or the way the
organization handles a crisis situation. Conduct-
ing research to better understand how these rela-
We wish to thank Frederick L. Oswald for his thoughtful
comments on a draft of this manuscript.
tional processes develop and unfoldis challenging.
One-time questionnaire administrations to organi-
zational members are notably decient for study-
ing such phenomena as this approach fails to ac-
count for the complexities of the dynamic na-
ture of relational processes. Fortunately, promise
for investigating relational processes is found in
the experience sampling methodology (ESM1;e.g.
Csikszentmihalyi and Graef, 1980; Csikszentmi-
halyi, Larson and Prescott, 2014). ESM provides
a powerful way for researchers to examine com-
plex, ecologically based research questions related
to short-term, within-person dynamic processes
1There are several variants of the ESM design, includ-
ing daily diary studies and the event-contingent recoding
method. We have chosen to use the term ESM in this pa-
per, though our comments and recommendationsare rel-
evant to the other designs as well.
© 2021 British Academy of Management and Wiley Periodicals LLC. Published by JohnWiley & Sons Ltd, 9600 Gars-
ington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA, 02148, USA.
554 E. D. Heggestad et al.
and to explore how those processes are inuenced
by changes in contextual factors (Ebner-Priemer
et al., 2009; Fisher and To, 2012). As such, the
use of the ESM allows management scholars to
directly examine how changes in contextual fac-
tors affect the moods, thoughts, perceptions and
behaviours of organizational members.
The repeated assessment of participants cre-
ates design and implementation challenges thatare
different from research methodologies that have
been more commonly employed in management
research. Fortunately, a series of methodologically
orientated papers have been written to help re-
searchers navigate decisions for designing and im-
plementing an ESM study – for example, see Beal
(2015), Bolger, Davis and Rafaeli (2003), Fisher
and To (2012), Gabriel, Podsakoff, Beal, Scott,
Sonnentag, Trougakos and Butts (2019), Hektner,
Schmidt and Csikszentmihalyi (2007), Ohly et al.
(2010), Scollon, Prieto and Diener (2009) and Uy,
Foo and Aguinis (2010). These papers provide ex-
cellent discussions and recommendations for re-
searchers looking to design and carry out an ESM
study.
The unique elements of the ESM also require
authors to provide different or, in some cases,
more elaborated information about aspects of
their study than they might when writing about
research based on other designs. This is particu-
larly true around descriptions of the sample and
of the measures given, as these are clear points of
departure from more commonly employed meth-
ods. Yet, there has not been a movement towards
establishing a set of reporting standards for ESM
studies to date. We hope to begin that movement
in this paper. Establishing reporting standards for
ESM research would be valuable for at least four
reasons, including: (1) enhanced transparency of
the science; (2) providing necessary information
for future meta-analytic work; (3) understanding
and evaluating replicationstudies; and (4) an over-
all improvement in the quality of our science.
In this paper, we provide recommendations for
the reporting of sample- and measurement-related
information in manuscripts describing ESM re-
search. Our reporting recommendations are in-
formed by sampling theory, measurement the-
ory, best practice recommendations for carrying
out ESM studies (e.g. Gabriel et al., 2019; Ohly
et al., 2010) and discussions of how to adapt, or
change, scales to t particular research contexts
(e.g. Heggestad et al., 2019). Our recommenda-
tions were further informed by comparing these
conceptual foundations for reporting sample- and
measurement-related information with current re-
porting conventions within the ESM literature.
Specically, we examined a sample of 110 pa-
pers (118 individual studies) published in the
management-related journals that employed an
ESM design to identify current reporting conven-
tions (i.e. what information was typically being re-
ported in describing samples and measures). (A
complete description of our methodology for col-
lecting these reporting convention data, as well
as descriptive information for the sample, can be
found in the online supporting information.) Con-
sidering these reporting conventions with respect
to the conceptual foundations allowed us to iden-
tify areas where essential information was either
presented ambiguously or not reported consis-
tently. Our recommendations are constructed to
carefully capture the information identied in our
conceptual review and to correct or clarify issues
we found in our survey of current reporting con-
ventions. As such, the purpose of this paper is
not to advise researchers on how best to conduct
ESM research, but rather to provide practical, eas-
ily implemented recommendations for the nature
and type of sample- and measurement-related in-
formation that should be reported in manuscripts
describing ESM research.
Our paper is structured as follows. First, we
provide a discussion of the distinctions between
reporting conventions, recommended reporting
standards and formal reporting standards. Sec-
ond, we look at issues related to sample report-
ing. Here, we bring together the conceptual foun-
dations regarding whatsample information should
be reported and with what we identied as report-
ing conventions in the ESM literature. Building
on this information, we provide a recommenda-
tion for a table that captures key sample-related
information, highlighting the elements of the table
through the presentationof a detailed ctitious ex-
ample. Third, we discuss twomeasurement-related
issues, the adaptation of scales for use in ESM
studies and reliability estimation for the resulting
multi-level data. We again bring together the con-
ceptual foundations and currentreporting conven-
tions related to each of these issues to offer report-
ing recommendations. We provide a sample ap-
pendix (Appendix A) for clearly reporting scales
that are adapted for use in the study, again using a
ctitious research project.
© 2021 British Academy of Management and Wiley Periodicals LLC.
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting