Single‐ and Multiple‐Informant Research Designs to Examine the Human Resource Management−Performance Relationship
| Author | Inmaculada Beltrán‐Martín,Ana Belén Escrig‐Tena,Juan Carlos Bou‐Llusar,Vicente Roca‐Puig |
| Published date | 01 July 2016 |
| DOI | http://doi.org/10.1111/1467-8551.12177 |
| Date | 01 July 2016 |
British Journal of Management, Vol. 27, 646–668 (2016)
DOI: 10.1111/1467-8551.12177
Single- and Multiple-Informant Research
Designs to Examine the Human Resource
Management−Performance Relationship
Juan Carlos Bou-Llusar, Inmaculada Beltr´
an-Mart´
ın, Vicente Roca-Puig
and Ana Bel´
en Escrig-Tena
Department of Business Management and Marketing at Universitat Jaume I, Facultad de
Ciencias Jur´
ıdicas y Econ´
omicas, 12071 Castell´
on, Spain
Corresponding author email: ibeltran@emp.uji.es
During the last decades, many empirical studies have analysed the relationship between
human resource management and firm performance. Despite the call for multiple-rater
designs, a relatively large number of researchersstill rely on survey responses provided by
a single informant in each organization. Single-informant designs suer from a number
of problems, especially when the responses provided by dierent types of raters across
firms are pooled into a single dataset prior to assessing their equivalence across raters.
Using an illustration of the relationship between high performance work systems and
firm performance, in this paper we observe that responses provided by managers holding
dierent positions (human resourcemanagers and sales managers) dier significantly and
thereforepooling their responses into a single dataset may result in confusing conclusions.
Furthermore, we demonstrate that dierences arise in the estimated parameters when a
multiple-key-informantapproach, compared to a single-informant design, is adopted. For
these reasons, data collection using multiple keyinformants is recommended, based on the
assumption that some raters in the firm will be more knowledgeable about the variables
of interest than others.
Introduction
A crucial question in the human resource man-
agement (HRM) literature is the analysis of the
relationship between the human resource strat-
egy of the firm and organizational performance
(Appelbaum et al., 2000; MacDue, 1995; Youndt
et al., 1996). Although most of the empirical stud-
ies have found a positive association between the
two measures (Combs et al., 2006), the method-
ological limitations raised by several authors (e.g.
Gerhart, Wright and McMahan, 2000; Gerhart
et al., 2000; Huselid and Becker, 2000) suggest
Research supported by University Jaume I (Grant ref.
P1.1B2013-26), by the Spanish Ministry of Science and
Innovation (Ref. ECO2015-66671-P) and by Generalitat
Valenciana (Grant ref.AICO/2015/029).
that the conclusions they draw are premature to
say the least. Our study aims to contribute to
the methodological debate in the HRM field by
exploring the following research questions.
First, over the past years several studies in
the HRM field have used data collected through
questionnaires administered to informants (one
informant per firm) whose positions vary across
organizations. For instance, this is the case of
studies that use data provided, without distinc-
tion, by HR managers or senior managers (e.g.
Park et al., 2003), by various sta members from
CEO to junior managers (Guthrie, 2001), by HR
managers, owners and senior managers (Harel
and Tzafrir, 1999), or even by unspecified respon-
dents (Delaney and Huselid, 1996). These studies
then pool responses to create a single dataset for
statistical analyses (Rungtusanatham et al., 2008)
© 2016 British Academy of Management. Published by John Wiley & Sons Ltd, 9600 Garsington Road, Oxford OX4
2DQ, UK and 350 Main Street, Malden, MA, 02148, USA.
Single- and Multiple-Informant Research Designs 647
so the information about HRM and performance
is provided by the HR managersin some cases and
by the senior or other managers in other cases.
However, it is important to take into account that
when there are systematic dierencesin survey rat-
ings depending on who providesthe answers (or, in
the words of Huselid and Becker (2000), when the
‘respondents matter’) the rater chosen to respond
to the survey becomes a critical issue. Conclusions
drawn from studies that pool responses obtained
from dierent types of raters should be analysed
with caution when measurement invariance is
not examined prior to pooling data (Rungtu-
sanatham et al., 2008). It is not our intention
to call into question the results of this research
stream but rather, in our first research question, to
empirically analyse the consequence of pooling re-
sponses provided by dierent types of informants
and to examine whether the proposed relation-
ships between HRM and performance vary
depending on the respondent chosen to assess the
variables.
Second, the traditional data collection strategy
employed in the HRM field assumes that a single
person is able to provide accurate information
about all the variables that refer to the whole
organization (Gerhart, Wright and McMahan,
2000). This approach increases the probability that
the relationships between HRM and performance
will be aected by common method variance
(CMV) (Podsako et al., 2003). A recommended
strategy in the HRM field to avoid the risk of
CMV is to collect data from multiple sources, i.e.
using various informants in each organization
to provide responses to dierent questions in
the same questionnaire. Multiple-source studies
assume that some raters are more knowledgeable
than others in assessing the measures of interest
(i.e. dierential accuracy assumption) (Huselid
and Becker, 2000). Consistent with Huselid and
Becker (2000) and Wright et al. (2001), we believe
that more attention should be paid to ensuring
that the most knowledgeable informants are used
in order to increase the validity of the measures
and to reduce the potential CMV. In our second
research question we analyse whether the relation-
ship between HRM and performance varies when
multiple informants in a single company assess the
variables aboutwhich they have more information
compared to a single-respondent survey design to
evaluate the same variables.
An overview of research approaches in
the HRM−performance literature
Many of the empirical studies in the HRM field
analysing the relationship between HRM and
performance use survey research. According to
Rungtusanatham et al. (2008), data in these stud-
ies are collected through dierent survey research
approaches, including those using either a single
or multiple informants. To provide a parsimo-
nious and systematic consideration of how pub-
lished articles on the HRM−performance rela-
tionship rely on single or multiple informants,
Chee (Appendix 1) shows a classification of 97
studies on this relationship included in Jiang et al.’s
(2012) meta-analysis, all of which used surveys
to collect all or part of the research data. A di-
rect content analysis (Hsieh and Shannon, 2005)
was conducted to classify the papers. Inspired
by the Weber protocol (Weber, 1990), the cod-
ing categories were defined based on Rungtu-
sanatham et al.’s (2008) classification approaches,
which were refined after coding a sample of the
selected articles. The reliability of the alloca-
tion of articles in each category was addressed
following the two-step procedure used in other
content analyses such as Furrer, Thomas and
Goussevskaia (2008), Jeung et al. (2011) and Clark
et al. (2014). First, two of the authors of this paper
independently reviewed all 97 selected articles and
coded them in one of the four categories (based on
a detailed examination of the methodology used);
the two authors coded 79% of the articles in the
same category. The percentage of agreement rose
to above90% when it was considered that some di-
vergences came from papers that shared features
from more than one approach, which the coders
had allocated in dierent categories. In a second
step, vagueness or discrepancies between the two
coders were resolved through research team dis-
cussions and assigned to the category considered
to be the best fit.
The papers were classified into four distinct
approaches. In the first approach (approach 1) a
single informant in each organization provides an-
swers to all the questions in the questionnaire, but
diers in that the position of the informants varies
across the organizations. For example, Audea,
Teo and Crawford (2005) used data obtained by
aggregating responses from HR managers, labour
or union representatives and general managers
© 2016 British Academy of Management.
To continue reading
Request your trial