A new classification system to describe the ageing of scientific journals and their impact factors
| Pages | 387-419 |
| Date | 01 October 1998 |
| Published date | 01 October 1998 |
| DOI | https://doi.org/10.1108/EUM0000000007175 |
| Author | H.F. Moed,Th. N. Van Leeuwen,J. Reedijk |
| Subject Matter | Information & knowledge management,Library & information science |
Journal of Documentation, Vol. 54, No. 4, September 1998
© Aslib, The Association for Information Management.
All rights reserved. Except as otherwise permitted under the Copyright, Designs and Patents Act
1988, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in
any form or by any means, electronic, mechanical, photocopying or otherwise without the prior
written permission of the publisher.
Aslib, The Association for Information Management
Staple Hall, Stone House Court, London EC3A 7PB
Tel: +44 (0) 171 903 0000, Fax: +44 (0) 171 903 0011
Email: pubs@aslib.co.uk, WWW: http://www.aslib.co.uk/aslib
A NEW CLASSIFICATION SYSTEM TO DESCRIBE THE AGEING OF
SCIENTIFIC JOURNALS AND THEIR IMPACT FACTORS
H.F. MOED+
moed@cwts.leidenuniv.nl
TH. N. VAN LEEUWEN+
leeuwen@cwts.leidenuniv.nl
J. REEDIJK++
reedijk@chem.leidenuniv.nl
+Centre for Science and Technology Studies (CWTS), Leiden University
PO Box 9555, 2300 RB Leiden, The Netherlands
++Leiden Institute of Chemistry, Leiden University, PO Box 9502
2300 RA Leiden, The Netherlands
During the past decades, journal impact data obtained from the
Journal Citation Reports (JCR) have gained relevance in library
management, research management and research evaluation. Hence,
both information scientists and bibliometricians share the responsi-
bility towards the users of the JCR to analyse the reliability and
validity of its measures thoroughly, to indicate pitfalls and to
suggest possible improvements. In this article, ageing patterns are
examined in ‘formal’ use or impact of all scientific journals
processed for the Science Citation Index (SCI) during 1981–1995. A
new classification system of journals in terms of their ageing charac-
teristics is introduced. This system has been applied to as many as
3,098 journals covered by the Science Citation Index. Following an
earlier suggestion by Glänzel and Schoepflin, a maturing and a
decline phase are distinguished. From an analysis across all subfields
it has been concluded that ageing characteristics are primarily
specific to the individual journal rather than to the subfield, while the
distribution of journals in terms of slowly or rapidly maturing or
declining types is specific to the subfield. It is shown that the cited
half life (CHL), printed in the JCR, is an inappropriate measure of
decline of journal impact. Following earlier work by Line and
387
THE
Journal of Documentation
VOLUME 54 NUMBER 4 SEPTEMBER 1998
387
Journal of Documentation, vol. 54, no. 4, September 1998, pp. 387–419
others, a more adequate parameter of decline is calculated taking
into account the size of annual volumes during a range of fifteen
years. For 76% of SCI journals the relative difference between this
new parameter and the ISI CHL exceeds 5%. The current JCR jour-
nal impact factor is proven to be biased towards journals revealing
a rapid maturing and decline in impact. Therefore, a longer term
impact factor is proposed, as well as a normalised impact statistic,
taking into account citation characteristics of the research subfield
covered by a journal and the type of documents published in it.
When these new measures are combined with the proposed ageing
classification system, they provide a significantly improved picture
of a journal’s impact to that obtained from the JCR.
1. INTRODUCTION
Patterns in the impact or use of scientific literatures as a function of their age
are well known subjects for study in information science and in bibliometrics.
Particularly citation distributions of scientific journals constitute an important
topic. A distinction can be made between formal and informal use, being two dis-
tinct aspects of the role of scientific literatures in the communication system.
Journal citation studies are assumed to measure formal communication and use
[1]. A scientific journal has proven to be an appropriate unit of analysis [2].
Generally, a journal has a specific scope, is associated with a specific group of
publishing authors and reading audiences. Moreover, it has specific publication
characteristics such as the number of issues and documents published per year, its
publication delay – i.e. the time interval between submission date of an article and
its actual publication date – and its circulation.
The Institute for Scientific Information (ISI) publishes on a regular basis jour-
nal impact factors in the
Journal Citation Reports (JCR). The impact factor (IF)
of journal Xfor a particular year Tis defined as the number of citations received
in year Tby all documents published in Xin the years T-1 and T-2, divided by the
number of citable documents published in Xin the years T-1 and T-2 [2]. In other
words, the IF measures the average impact in a year, of documents published in a
journal one or two years earlier. The impacts of older documents are completely
ignored. Hence this IF should be qualified as a short term impact factor. One of
the basic assumptions underlying the IF is that the number of citations given to a
journal is proportional to the number of citable documents in that journal. In order
to correct for differences in the size of the annual volumes among journals, the
impact factor is expressed as a ratio of the number of citations collected and the
number of citable documents published in the journal.
During the past decades, JCR journal impact factors gained relevance for var-
ious user populations. First, they are applied by librarians as tools for the man-
agement of their journal collections, often in combination with other types of
information, such as actual library uses [3]. A second group consists of scientists
who use impact factors for the evaluation and redirection of their publication
strategies [4]. Finally, impact factors are more and more used in the assessment of
research performance of individual scientists, research groups, institutes, univer-
sities or even countries. As pointed out by van Raan and several other authors,
bibliometric indicators have proven to be useful tools in the assessment of
JOURNAL OF DOCUMENTATION vol. 54, no. 4
388
Journal of Documentation, Vol. 54, No. 4, September 1998
© Aslib, The Association for Information Management.
All rights reserved. Except as otherwise permitted under the Copyright, Designs and Patents Act
1988, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in
any form or by any means, electronic, mechanical, photocopying or otherwise without the prior
written permission of the publisher.
Aslib, The Association for Information Management
Staple Hall, Stone House Court, London EC3A 7PB
Tel: +44 (0) 171 903 0000, Fax: +44 (0) 171 903 0011
Email: pubs@aslib.co.uk, WWW: http://www.aslib.co.uk/aslib
research performance, provided that such indicators have a sufficient level of
sophistication, that their pitfalls are taken into account, and that they are used in
combination with other, more qualitative information on the research units
analysed [5].
In evaluation procedures, ISI impact factors – however inaccurate they may be
[6] – are assumed to indicate the impact or utility of scientific journals [2].
University librarians are frequently requested by scientists, science evaluators or
policy makers to provide copies of the section ‘Journals Ranked by Impact
Factor’ from the JCR. As an alternative to ‘simple’ publication counts, in assess-
ments of research groups, articles are weighted by the impact factor of the jour-
nals in which they were published. Given this practical use of journal impact data
obtained from the JCR, information scientists and bibliometricians share the
responsibility towards the users of the JCR to conduct careful, thorough and accu-
rate analyses of the reliability and validity of its measures, to indicate pitfalls and
to suggest possible improvements. It is from this concern that the current article
is written.
Several authors pointed towards the problem that the ISI journal impact factor,
as it measures the impact of articles obtained one or two years after publication
date, may be affected by ageing characteristics. Glänzel and Schoepflin issued a
warning that ISI journal impact factors may be distorted by deviating ageing
behaviour, even in a comparative analysis of journals covering the same subfield
[7]. As long as ten years ago Rousseau suggested calculating impact factors based
on a five year observation period rather than the two year period applied by ISI in
the field of mathematics [8].
For a limited number of journals, the maximum average impact of its docu-
ments is reached two years after publication date. For these journals, the ISI
impact factor covers the ‘best’ year in terms of impact. However, for other jour-
nals this maximum is obtained three, four, five or even more years after publica-
tion date. Consequently, for such journals the ‘best’ year in terms of impact is not
included in the presently used IF. It is on this problem that our article has its main
focus. To what extent are ISI journal impact factors affected by ageing character-
istics? To what extent are they biased towards journals revealing specific ageing
patterns, particularly those related to a high ‘immediacy’ value of a journal?
The producer of the JCR seems to be aware of this problem, as it suggests in
an assessment of the impact of scientific journals that a second measure printed
for each journal in the JCR, denoted as cited half life (CHL) should be taken into
account. This measure is estimated for each citing year, and is defined as ‘the
number of journal publication years going back from the current year which
account for 50% of the total citations received by the cited journal in the current
year’. We will show that this parameter insufficiently reflects ageing characteris-
tics of a journal, and we will present more appropriate indicators for this ageing.
An important distinction in the literature on ageing is between a synchronous
and a diachronous approach. The first approach deals with age distributions of
citations given by constant user (citing) populations, while in the second the user
(citing) population varies, and the year of the cited articles is fixed. According to
the terminology adopted by Line and Sandison [9], a diachronous approach deals
with date distributions rather than age distributions. Glänzel and Schoepflin
September 1998 JOURNAL IMPACT DATA
389
Journal of Documentation, Vol. 54, No. 4, September 1998
© Aslib, The Association for Information Management.
All rights reserved. Except as otherwise permitted under the Copyright, Designs and Patents Act
1988, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in
any form or by any means, electronic, mechanical, photocopying or otherwise without the prior
written permission of the publisher.
Aslib, The Association for Information Management
Staple Hall, Stone House Court, London EC3A 7PB
Tel: +44 (0) 171 903 0000, Fax: +44 (0) 171 903 0011
Email: pubs@aslib.co.uk, WWW: http://www.aslib.co.uk/aslib
To continue reading
Request your trial