Mapping research activities and societal impact by taxonomy of indicators: uniformity and diversity across academic fields
| Date | 23 January 2023 |
| Pages | 1049-1070 |
| DOI | https://doi.org/10.1108/JD-06-2022-0131 |
| Published date | 23 January 2023 |
| Subject Matter | Library & information science,Records management & preservation,Document management,Classification & cataloguing,Information behaviour & retrieval,Collection building & management,Scholarly communications/publishing,Information & knowledge management,Information management & governance,Information management,Information & communications technology,Internet |
| Author | Marianne Lykke,Louise Amstrup,Rolf Hvidtfeldt,David Budtz Pedersen |
Mapping research activities
and societal impact by taxonomy
of indicators: uniformity and
diversity across academic fields
Marianne Lykke
Department of Communication and Psychology, Aalborg Universitet,
Aalborg, Denmark, and
Louise Amstrup, Rolf Hvidtfeldt and David Budtz Pedersen
Department of Communication and Psychology, Aalborg University,
Copenhagen, Denmark
Abstract
Purpose –Several frameworks have been developed to map and document scientific societal interaction and
impact, each reflecting the specific forms of impact and interaction that characterize different academic fields.
The ReAct taxonomy was developed to register data about “productive interactions”and provide an overview
of research activities within the social sciences and humanities (SSH). The purpose of the present research is to
examine whether the SSH-oriented taxonomy is relevant to the science, technology, engineering and
mathematics (STEM) disciplines when clarifying societal interactions and impact, and whether the taxonomy
adds value to the traditional STEM impact indicators such as citation scores and H-index.
Design/methodology/approach –The research question was investigated through qualitative interviews
with nine STEM researchers. During the interviews, the ReAct taxonomy and visual research profiles based on
the ReAct categories were used to encourage and ensure in-depth discussions. The visual research profiles were
based on publicly available material on the research activities of the interviewees.
Findings –The study providedan insight into how STEM researchers assessed the importanceof mapping
societal interactions as a background for describing research impact,including which indicators are usefulfor
expressing societal relevance and impact. With regard to the differences between STEM and SSH, the study
identifieda high degree of cohesionand uniformity in theimportance of indicators.Differences were moreclosely
related to the purpose of mapping and impact assessment than between scientific fields. The importance of
amalgamationand synergy between academic andsocietal activities was also emphasised and clarified.
Practical implications –The findings highlight the importance of mapping societal activities and impact,
and that societal indicators should be seen as inspiring guidelines depending on purpose and use. A significant
contribution is the identification of both uniformity and diversity between the main fields of SSH and STEM, as
well as the connection between the choice of indicators and the purpose of mapping, e.g. for impact
measurement, profiling, or career development.
Originality/value –The work sheds light on STEM researchers’views on research mapping, visualisation
and impact assessment, including similarities and differences between STEM and SSH research.
Keywords Research mapping, Research impact, Research evaluation, Research information management,
Societal impact, Societal interactions, STEM research
Paper type Research paper
1. Introduction
There is increasing recognition in the current literature that academic fields require multiple
diverse frameworks for mapping, visualising and assessing research activities and impact
(Pedersen et al., 2020). Indicators must reflect how different disciplines are engaged in different
Taxonomy of
impact
indicators
1049
This work was supported by the Danish Agency for Science and Higher Education and the Obel Family
Foundation, grant no. 27954. The authors would also like to thank the interview participants that
contributed with valuable insight.
The current issue and full text archive of this journal is available on Emerald Insight at:
https://www.emerald.com/insight/0022-0418.htm
Received 16 June 2022
Revised 2 November 2022
Accepted 6 November 2022
Journal of Documentation
Vol. 79 No. 5, 2023
pp. 1049-1070
© Emerald Publishing Limited
0022-0418
DOI 10.1108/JD-06-2022-0131
areas of society, and therefore utilise a broad range of impact pathways (Lauronen, 2020). Rather
than synthesising different impact pathways under the same simplistic framework, it is necessary
to study how different disciplines are engaged in divergent practices of knowledge exchange, co-
creation and dissemination. While STEM disciplines have traditionally taken the lead by
establishing indicators for intellectual property, such as patents, royalties, licenses and contracts,
the same indicators do not apply to the same degree for SSH. Put simply, impact does not mean the
same thing across academic fields, institutions, geographic location and research cultures (Hill,
2016;Sivertsen and Meijer, 2020). This is not surprising. Different academic fields generate
different forms of impact in society and engage in different forms of interaction –ranging from
knowledge and technology transfer to a more informal transfer of knowledge, advice, data and
expertise. This complexity and diversity are reflected in the large number of methods and
frameworks used to demonstrate and assess the impact of research (Penfield et al.,2014;Pedersen
et al.,2020). Different frameworks incorporate diverse sets of indicators and methods to describe
research impact –ranging from academic impact to policy impact, social impact, educational
impact, cultural impact and economic impact (Pedersen et al.,2020). Both quantitative methods
(e.g. citation analysis and commercialisation data) and qualitative methods (e.g. case studies,
interviews, field visits and research information management systems) are regularly used to
describe different aspects of the impact lifecycle. Each method has its advantages and
disadvantages (Ferretti et al.,2018;Pedersen et al.,2020). Hicks et al. (2015) argue that indicat ors
should always be able to account for variation in specific research fields and protect locally
relevant research. Assessments must be open and transparent, and quantitative evaluations
should always support qualitative and expert assessments and vice versa.
Along the same lines, questions about how to register and validate research impact have
multiplied. Debates have arisen concerning the proper unit of analysis: the individual, the
research group, the institution, etc. Similarly, consequential debates about the notion of
impact itself are being argued across almost all academic institutions and funding agencies.
Related to these debates are ontological questions about which taxonomies and metadata are
required to register and describe the large diversity of academic activities and outcomes, as
well as methodological questions about the nature of evaluation. Whereas some frameworks
have opted for an “attribution analysis”, i.e. casually attributing change of practice in society
to research output, other models have focused on “contribution analysis”, i.e. acknowledging
that impact is often generated through multiple contributing factors and is highly dependent
on social readiness and absorptive capacity. Contribution analysis (or contribution mapping)
has led to the creation of several impact frameworks that perceive the impact of research in
society as the result of “productive interactions”between (units of) research organisations
and other public and private organisations, which eventually leads to an effect or “impact”
outside of the research environment (Molas-Gallart and Tang, 2011;Spaapen and van Drooge,
2011;Bornmann, 2013;D’Este et al., 2018;Pedersen et al., 2020;Sivertsen and Meijer, 2020).
The framework of “productive interactions”has become useful in understanding the large
diversityofimpactarisingfromdifferentdisciplines. Rather than focusing on specific outcomes or
effects, the framework suggests that productive interactions between research and society can
emerge at different levels and in different contexts. Originally, the SIAMPI (Social Impact
Assessment Methods through Productive Interactions) focused on interactions in three
dimensions: direct interactions, indirect interactions and financial interactions (Spaaden and
van Drooge, 2011). However, in principle, the methodology can be extended to cover other
interactions that may be deemed useful for studying knowledge exchange and impact pathways.
This shift in focus from attribution to contribution analysis has several advantages. The
framework of productive interactions does not attempt to identify evidence of impact but focuses
on the process of interaction and co-creation to trace thepathwaysandinteractionsthatmaylead
to impact in society. By focusing on activities that facilitate exchange, communication, translation
and implementation of research in different organisations, Sivertsen and Meijer (2020, p. 68)
JD
79,5
1050
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting