The culture of orphaned texts. Academic books in a performance-based evaluation system

Date19 November 2018
Pages623-642
Published date19 November 2018
DOIhttps://doi.org/10.1108/AJIM-03-2018-0063
AuthorLudek Broz,Tereza Stöckelová
Subject MatterLibrary & information science,Information behaviour & retrieval,Information & knowledge management,Information management & governance,Information management
The culture of orphaned texts
Academic books in a performance-based
evaluation system
Ludek Broz
Institute of Ethnology of the Czech Academy of Sciences, Prague,
Czech Republic, and
Tereza Stöckelová
Institute of Sociology of the Czech Academy of Sciences, Prague, Czech Republic
Abstract
Purpose The purpose of this paper is to contribute to the body of knowledge on how research evaluation in
different national and organisational contexts affects, often in unintended ways, research and publication
practices. In particular, it looks at the development of book publication in the social sciences and humanities
(SSH) in the Czech Republic since 2004, when a performance-based system of evaluation was introduced, up to
the present.
Design/methodology/approach The paper builds upon ethnographic research complemented by the
analysis of Czech science policy documents, data available in the governmental database Information
Register of R&D resultsand formal and informal interviews with expert evaluators and other stakeholders
in the research system. It further draws on the authorsown experience as scholars, who have also over the
years participated in a number of evaluation procedures as peers and experts.
Findings The number of books published by researchers in SSH based at Czech institutions has risen
considerably in reaction to the pressure for productivity that is inscribed into the evaluation methodology and
has resulted in the rise of in-house publishing by res earchersown research institution, fa ke
internationalisationusing foreign low-quality presses as the publication venue, and the development of a
culture of orphaned books that have no readers.
Practical implications In the Czech Republic robust and internationally harmonised bibliometric data
regarding books would definitely help to create a form of research evaluation that would stimulate
meaningful scholarly book production. At the same time, better-resourced and better-designed peer review
evaluation is needed.
Originality/value This is the first attempt to analyse in detail the conditions and consequences the Czech
performance-based research evaluation system has for SSH book publication. The paper demonstrates that
often discussed harming of SSH and book-writing in particular by performance-based IF-centred research
evaluation does not necessarily manifest in declining numbers of publications. On the contrary, the number of
books published may increase at the cost of producing more texts of questionable scholarly quality.
Keywords Czech Republic, Research evaluation, Classification, Book publication,
Social sciences and humanities (SSH), Vanity publishing
Paper type Research paper
1. Introduction
Judging from statistics maintained by the state, the year 2008 was a successful one in the
social sciences and humanities (SSH) in the Czech Republic. The number of books registered
that year as scientific output in those disciplines doubled when compared with the year
2007. In absolute numbers, SSH researchers working for Czech institutions listed 1484 books
in 2008, while in 2007 it was 749. This sudden jumpwas partly an effect of research
institutions registering books published earlier, something that also explains the two
sudden increases in 1998 and 2001. But while the earlier two increases were followed by
Aslib Journal of Information
Management
Vol. 70 No. 6, 2018
pp. 623-642
© Emerald PublishingLimited
2050-3806
DOI 10.1108/AJIM-03-2018-0063
Received 16 March 2018
Revised 20 August 2018
11 October 2018
Accepted 12 October 2018
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/2050-3806.htm
The work of Tereza Stöckelová on the paper was supported by Grant No. 15-16452S awarded by the
Czech Science Foundation. The authors would like to thank Filip Vostal, Sarah de Rijcke and
the two anonymous reviewers for valuable comments on earlier versions of the paper and
Šárka Loffelmannová for helping with the RIV data preparation.
623
The culture of
orphaned texts
decreases, the one in 2008 resulted in a permanent rise in production. If we look at the
figures based on the year of publication (rather than the year of registration in the system)
we find a visible but more gradual trend of growing numbers of books being produced in
Czech SSH (Figure 1).
This trend correlates with the gradual implementation of a performance-based system of
research evaluation in the Czech Republic since 2004. In this paper we explore the systemic
conditions and consequences of this at first glance paradoxical increase of productivity in book
writing and publishing, which arguably represents the most conservative, labour-intensive and
time-consuming academic activity. We thereby seek to contribute to the body of lessons
(Bonaccorsi, 2018a) that can be learned from the various national research evaluation systems
that have been introduced in continental Europe over the past two decades, in different forms, at
different speeds, and at different stages of a research systems development. We believe detailed,
ethnographically informed studies are much needed as a starting point for a comparative
empirical study of evaluative cultures(Lamont and Guetzkow, 2016, p. 40), which is in turn
necessary if there is to be any meaningful revision of evaluation methodology and the
development of bibliometric approaches to book publication.
Performance-based IF-centred exercises in research evaluation are often criticised for
harming SSH and book-writing in particular (see, e.g. Ochsner et al., 2016). In this paper we
demonstrate that this harm does not necessarily mean the number of books decreases.
On the contrary, it means that the number of books published may increase but at the cost of
producing more texts of questionable scholarly quality. This may be one of the many
problematic and unwelcome consequences of the specific parameters of Czech assessment
methodology (Good et al., 2015; Linková and Stöckelová, 2012; Stöckelová, 2012; Macháček
and Srholec, 2017; Stöckelová and Vostal, 2017).
After briefly introducing the history and current state of the Czech research evaluation
system, we focus primarily on books and their changing definition and valuation in the
assessment methodology. We proceed by discussing the key, largely negative consequences
of the specific parameters of the system of book publishing in the SSH, which include the
rise of research institutions doing in-house publishing, fake internationalisation, that is,
0
200
400
600
800
1,000
1,200
1,400
1,600
1,800
2,000
1990 1995 2000 2005 2010 2015 2020
Number of books
Notes: Dashed line represents number of books according to the year of registration.
Solid line represents number of books according to the year of publication (Data for the
year 2017 is not yet fully incorporated). The RIV database keeps changing even
retrospectively, the graph reflects figures as retrieved by the authors in January 2018
from www.rvvi.cz/riv?s=rozsirene-vyhledavani
Figure 1.
Books in SSH
published by
researchers based at
Czech institutions
according to the
information Register
of R&D results (RIV)
624
AJIM
70,6

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT