Reordering Our Priorities by Putting Phenomena before Design: Escaping the Straitjacket of Null Hypothesis Significance Testing
| DOI | http://doi.org/10.1111/1467-8551.12063 |
| Author | Abagail McWilliams,Andy Lockett,David D. Van Fleet |
| Date | 01 October 2014 |
| Published date | 01 October 2014 |
Methodology Corner
Reordering Our Priorities by Putting
Phenomena before Design: Escaping the
Straitjacket of Null Hypothesis
Significance Testing
Andy Lockett, Abagail McWilliams1and David D. Van Fleet2
Warwick Business School, Coventry CV4 7AL, UK, 1College of Business Administration, University of
Illinois at Chicago, 601 S. Morgan St (MC 243), Chicago, IL 60607-7123, USA, and 2Morrison School of
Agribusiness, W. P. Carey School of Business, Arizona State University, Mesa, Arizona, USA
Emails: andy.lockett@wbs.ac.uk; abby@uic.edu; ddvf@asu.edu
In this paper we reflect on two related questions. First, how have we arrived at a position
where null hypothesis significance testing is the dominant criterion employed by quan-
titative researchers when deciding on whether or not a result is ‘significant’? Second, how
might we change the practice of quantitative management research by promoting a
greater plurality of methods, and in doing so better enable scholars to put phenomena
before design? We conclude by arguing that quantitative management researchers need
to focus on the epistemological issues surrounding the role of scholarly reasoning in
justifying knowledge claims. By embracing a plurality of approaches to reasoning quan-
titative researchers will be better able to escape the straitjacket of null hypothesis
significance testing and, in doing so, reorder their priorities by putting phenomena before
design.
Judge a person by their questions, rather than
their answers. (Voltaire)
In this paper we explore the relationship between
the phenomena and design of research, with a
specific focus on the application of null hypothesis
significance testing (NHST) in management
research. We do so because the study of the prac-
tice of research matters as ‘research findings often
say more about a researcher’s practices than the
phenomena studied’ (Starbuck, 2004, p. 1). Man-
agement researchers face the dilemma that many
of our existing practices mimic those in the
natural sciences but still fail to further our under-
standing of the world (Starbuck, 2004). This
places us in the unenviable position of producing
studies that may have a detrimental impact on
management practice and on future management
research if/when the mimicking of natural science
methods is inappropriate (Ghoshal, 2005).
NHST is an interesting case in point of this
possibility. Originally developed by Fisher to
provide an ‘objective’ methodology for experi-
ments in fundamental research, where little was
known about potential effects, NHST was
designed as a test for whether or not there is an
effect; i.e. it is a pure test of the mere existence of
an effect. Although NHST has a veneer of scien-
tific rigour, commentators have detailed the prob-
lems of employing NHST across a range of
different social science disciplines including
We would like to thanks Saku Mantere for his invaluable
advice on early versions of this manuscript, and three
anonymous reviewers for their insights and improve-
ments during the review process.
bs_bs_banner
British Journal of Management, Vol. 25, 863–873 (2014)
DOI: 10.1111/1467-8551.12063
© 2014 British Academy of Management. Published by John Wiley & Sons Ltd, 9600 Garsington Road, Oxford OX4
2DQ, UK and 350 Main Street, Malden, MA, 02148, USA.
To continue reading
Request your trial