Business and Management Impact Assessment in Research Excellence Framework 2014: Analysis and Reflection
| Date | 01 October 2016 |
| Published date | 01 October 2016 |
| DOI | http://doi.org/10.1111/1467-8551.12186 |
British Journal of Management, Vol. 27, 693–711 (2016)
DOI: 10.1111/1467-8551.12186
Business and Management Impact
Assessment in Research Excellence
Framework 2014: Analysis and Reflection
Neil M. Kellard and Martyna ´
Sliwa
Essex Business School, University of Essex, Wivenhoe Park,Colchester, CO4 3SQ, UK
Emails: nkellard@essex.ac.uk, masliwa@essex.ac.uk
The evaluation of research impact is likely to remain an important element of research
quality audits in the UK for the foreseeable future. With this paper, we contribute to
debates on impact and relevance of business and management studies research through
an analysis of Research Excellence Framework 2014 impact scores within the business
and management unit of assessment. We oer insights into the organizational contexts
of UK business schools within which impact is produced, drawingattention to the issues of
linkages with researchintensity, grant income generation, research team size, careerstage
and gender of academics, and whether impact activity is focusedon private or public sector
organizations and national or international reach. We put forwardrecommendations for
managers responsible for business schools and higher education policymakers regarding
management and organizational policies and processes, as well as possible changes to the
rules guiding future research excellenceaudits.
Introduction
Since the first Research Assessment Exercise in
1986, the idea of evaluating universityresearch has
been well established in the UK system of higher
education. The Research Excellence Framework
2014 (REF2014) brought the first exercise in na-
tional assessment of research that focused on the
evaluation not only of research quality but also
of its impact outside academia. In preparing for
REF2014, this newly added emphasis on impact
had prompted UK higher education institutions
(HEIs) to pay greater attentionto evidencing their
engagement with external stakeholders and to the
ways in which the generation of impact is embed-
ded in the organizational culture and in existing
structures for supporting and managing research.
At present, British universities have already started
to plan for the next assessment of research quality,
commonly referred to as REF2020. At this point,
it is not certain what form the next assessment is
going to take but it would appear that evaluation
of research impact is here forthe foreseeable future.
In the case of business schools, the topic of
research impact has for a number of years now
attracted the attention of scholars contributing
to the ‘relevance debate’ (e.g. Butler, Delaney
and Spoelstra, 2015; Fincham and Clark, 2009;
Gulati, 2007; Hodgkinson and Starkey, 2011;
Kieser, Nicolai and Seidl, 2015; Pettigrew, 2011).
While the debate has addressed a range of issues,
from questioning how to define relevance (e.g.
Willmott, 2012) to discussing how to combine
academic rigour with relevance to practice within
business and management studies (BMS) research
(e.g. Hodgkinson and Starkey, 2011), little has
been said about the organizational conditions and
contexts in which BMS research is carried out
and – with the exception of Butler, Delaney and
Spoelstra’s (2015) recent piece with a focus on
leadership scholars – about the researchers who
produce ‘impactful research’.
© 2016 British Academy of Management. Published by John Wiley & Sons Ltd, 9600 Garsington Road, Oxford OX4
2DQ, UK and 350 Main Street, Malden, MA, 02148, USA.
694 N. M. Kellard and M. ´
Sliwa
Against this background, our paper contributes
to discussions of impact and relevance of business
school research. We address the UK higher edu-
cation sector in which these debates are interwo-
ven with considerations of the periodic audits of
university research carried out through REF exer-
cises. We are particularly interested in the insights
that REF2014 results provide for our understand-
ing of (1) what kind of business school contexts
are most conducive to generating research impact;
(2) who actually produces the type of impact and
the underpinning research that would attract high
grades from the REF assessors; and (3) whatimpli-
cations this has for business school managers and
for policymakers within the UK system of higher
education.
In exploring these questions, we have under-
taken an empirical analysis of impact scores within
the business and management unit of assessment
(UoA) with a focus on comparison between
three performance-related clusters of institutions.
A cluster-based comparison has allowed us to
concentrate on the more general issues of sim-
ilarities and dierences between organizational
contexts and the impact activity business schools
engage in, rather than on the detail of the content
of individual impact case studies. Against the
background of extant literature on the impact
and relevance of business school research, our
paper oers insights into the organizational con-
texts within which impact is produced, drawing
attention to the issues of linkages with research
intensity, grant income generation, research team
size, career stage and gender of academics, and
whether impact activity is focused on private
or public sector organizations and national or
international reach. Following from our analysis,
we put forward recommendations for managers
responsible for business schools and higher edu-
cation policymakers regarding management and
organizational policies and processes, as well
as possible changes to the rules guiding future
research excellence audits.
The remainder of this paper is structured as fol-
lows.First, we oer an overview of the literature on
impact and relevance of BMS research and point
to the importance of the so far under-explored
questions about the organizationalcontexts within
which impact is produced. We then provide a back-
ground to our analysis explaining key definitions
and requirements regarding research impact, as
stipulated in the REF guidelines issued in advance
of REF2014. This is followed by discussion of the
data and methods applied in the analysis. Subse-
quently, weoer an analysis of impact scores with
a focus on three clusters of institutions.Finally, we
discuss the findings of our analysis and provide a
set of conclusions.
Impact, relevance and the
organizational context of UK HEIs
In the Special Issue of this journal celebrating
its 25th anniversary, Andrew Pettigrew (2011,
p. 347) observed that ‘it may be appropriate for
some members of our local and international
communities to raise their aspirations and deliver
forms and processes of knowledge which meet
the double hurdle of scholarly quality and pol-
icy/practice impact’. At the same time, he noted
that management research suers from the lack
of ‘a natural focused community’(Pettigrew, 2011,
p. 349) comprising its recipients. These comments
were published soon after the Higher Education
Funding Council for England’s (HEFCE, 2011)
confirmation that in the next national exercise of
academic research evaluation (i.e. REF2014) there
would be an explicit element to assess the impact
arising from excellent research, alongside the out-
puts and environmental elements.
While being an advocate of business and man-
agement ‘research with impact’, back in 2011
Pettigrew pointed out that the eects of succes-
sive research evaluation assessments in the UK –
which he links to Power’s (1999) notion of the ‘au-
dit society’ – had been both positive and negative.
This is because, similarly to all evaluation mech-
anisms, rather than simply fulfilling the role of
measuring and ranking the research carried out by
academics, national assessments of research aect
in various ways the processes they evaluate, and
give rise to variegated outcomes at both the orga-
nizational and individual level (Smith, Ward and
House, 2011).
An espoused intention behind introducing the
impact element into REF2014 was to incentivize
behaviours that would shift the notion of aca-
demic research away from ‘classical conceptions of
knowledge conducted by elites’ (Hazelkorn, 2009,
p. 9) and towards rewarding research that focuses
on dierent stakeholders and benefits wider soci-
ety. As such, the logic behind placing importance
on the impact of research produced within the UK
© 2016 British Academy of Management.
To continue reading
Request your trial