“Let the community decide”? The vision and reality of soundness-only peer review in open-access mega-journals
| Pages | 137-161 |
| Date | 08 January 2018 |
| DOI | https://doi.org/10.1108/JD-06-2017-0092 |
| Published date | 08 January 2018 |
| Author | Valerie Spezi,Simon Wakeling,Stephen Pinfield,Jenny Fry,Claire Creaser,Peter Willett |
| Subject Matter | Library & information science,Records management & preservation,Document management,Classification & cataloguing,Information behaviour & retrieval,Collection building & management,Scholarly communications/publishing,Information & knowledge management,Information management & governance,Information management,Information & communications technology,Internet |
“Let the community decide”?
The vision and reality of
soundness-only peer review in
open-access mega-journals
Valerie Spezi
LISU, Centre for Information Management,
School of Business and Economics, Loughborough University, Loughborough, UK
Simon Wakeling and Stephen Pinfield
Information School, The University of Sheffield, Sheffield, UK
Jenny Fry
Department of Information Science, Loughborough University, Loughborough, UK
Claire Creaser
LISU, Loughborough University, Loughborough, UK, and
Peter Willett
Information School, The University of Sheffield, Sheffield, UK
Abstract
Purpose –The purpose of this paper is to better understand the theory and practice of peer review in
open-access mega-journals (OAMJs). OAMJs typically operate a “soundness-only”review policy aiming to
evaluate only the rigour of an article, not the novelty or significance of the research or its relevance to a
particular community, with these elements being left for “the community to decide”post-publication.
Design/methodology/approach –The paper reports the results of interviews with 31 senior publishers
and editors representing 16 different organisations, including 10 that publish an OAMJ. Thematic analysis
was carried out on the data and an analytical model developed to explicate their significance.
Findings –Findings suggest that in reality criteria beyond technical or scientific soundness can and do
influence editorial decisions. Deviations from the original OAMJ model are both publisher supported (in the
form of requirements for an article to be “worthy”of publication) and practice driven (in the form of some
reviewers and editors applying traditional peer review criteria to OAMJ submissions). Also publishers believe
post-publication evaluation of novelty, significance and relevance remains problematic.
Originality/value –The study is based on unprecedented access to seniorpublishers and editors, allowing
insight into their strategic and operational priorities. Thepaper is the first to report in-depth qualitative data
relating specifically to soundness-onlypeer review for OAMJs, shedding new lighton the OAMJ phenomenon
and helping informdiscussion on its future role in scholarlycommunication. The paper proposes a new model
for understandingthe OAMJ approachto quality assurance, andhow it is different from traditionalpeer review.
Keywords Economics, Electronic publishing, Electronic journals, Publishing, Free publications,
Journal publishers
Paper type Research paper
1. Introduction
Peer review is a fundamental element of scholarly publishing. The process, which involves
research papers submitted to journals being assessed by experts in the field in order to
inform the decision to accept or reject papers and improve accepted ones, is seen as an
Journal of Documentation
Vol. 74 No. 1, 2018
pp. 137-161
Emerald Publishing Limited
0022-0418
DOI 10.1108/JD-06-2017-0092
Received 20 June 2017
Revised 19 August 2017
Accepted 27 August 2017
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0022-0418.htm
© Valerie Spezi, Simon Wakeling, Stephen Pinfield, Jenny Fry, ClaireCreaser and Peter Willett. Published by
Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0)
licence. Anyone may reproduce, distribute, translate and create derivative works of this article ( for both
commercial and non-commercial purposes), subject to full attribution to the original publication and authors.
The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
137
Soundness-
only peer
review in
OAMJs
essential form of quality assurance (Procter et al., 2010; Rowland, 2002). Organised by
publishers and editors, and performed voluntarily by fellow researchers, it is an important
instrument of self-regulation for the scholarly community in conducting academic research
(Bohannon, 2013). In conventional journals, the peer review process takes place prior to
publication and is based on a number of commonly accepted criteria applied in the selection
of papers, including the rigour or soundness of the study, its novelty, its significance and its
relevance to the journal’s readership. Journals vary in their selectiveness, which for the most
part is a result of the thresholds applied to these criteria.
Since the web has been adopted as the primary means of disseminating scholarly content,
new forms of journal publishing, often involving innovative approaches to peerreview, have
emerged. Notable amongst these are so-called “mega-journals”(OAMJs), which include
PLOS ONE (the first journal of its kind), launched by the fully open-access (OA) publisher
Public Library of Science in 2006, and Scientific Reports, launched by Nature in 2011.
As journals published entirely online, these publications were freed from any
restrictions imposed by page space, a constraint partly responsible for selectivity in
conventional journals that originated in a print-based environment. Open-access
mega-journals (OAMJs) have four main characteristics: large publishing volume, broad
subject scope, an OA publishing model and a novel approach to peer review (Björk, 2015;
Spezi et al., 2017). The last of these requires editors and reviewers to evaluate only a paper’s
scientific or technical soundness, and not to take into account its novelty, significance
or its relevance to a notional readership. Evaluation of these remaining factors is
pushed downstream to be judged by the academic community after the article’s publication,
with indicators such as article-level metrics serving as supporting tools for such
post-publication assessment.
This decoupling of the judgement of novelty, significance and relevance from the
pre-publication peer review stage has been a controversial and much debated development
in scholarly publishing –a controversy fuelled by the launch of a large number of OAMJs in
the wake of PLOS ONE ’s success (Wakeling et al., 2016). Advocates claim that peer review
for soundness only has the potential to alter radically the traditional filtering and
gatekeeping role of journal editors, editorial boards and reviewers whose judgements on
novelty and significance have traditionally been crucial in determining what is published.
Judgements on novelty, significance and relevance are often criticised as being subjective,
and so peer review focussed on soundness only is argued to be more objective. One apparent
consequence of this is that editors, editorial boards and reviewers are no longer the sole
arbiters of the boundaries of existing disciplinary paradigms. As such, OAMJs can be
conceived of having democratising potential (Lăzăroiu, 2017). Advocates also argue that the
OAMJ approach to peer review allows for greater efficiency in the publication process by
eliminating the submission-rejection spiral undergone by many articles before they find a
home in a selective journal (Cope and Phillips, 2014; Pattinson, 2014; Ware, 2011), and in
allowing the publication of studies that might struggle to meet the significance
requirements of traditional journals –the reporting of negative results and replication
studies being often cited examples (Binfield, 2013). The approach can, therefore, be seen to
support the broader goals of the Open Science movement. Summarising the views of many
of its supporters, Gary Ward, Chair of the PLOS Board of Directors, was quoted in a 2011
interview as saying, “I love the concept of eliminating this huge waste of time by simply
removing subjective evaluations of importance from the review process […] If the paper is
well-written and the conclusions don’t overreach, then let the community decide the impact”
(Adams, 2011). That mantra of, “let the community decide”, has become a central feature of
the OAMJ rhetoric. On the other hand, soundness-only peer review has been criticised for
lowering acceptance thresholds in the interests of journal scalability and a highly lucrative
business model (Buriak, 2015).
138
JD
74,1
To continue reading
Request your trial