BMC Bioinformatics' delayed impact

Sun Nov 21 01:54:43 EST 2004

Dear Ronald and colleagues,

Why would one bring the main diagonal to zero in this case?

In my opinion, there are two components which are confused here:
self-citations and within-journal citations. Self-citations can also occur
across journal borders. Within-journal citations do not have to be
self-citations. Within-journal citations in most cases provide an outlier in
the distribution, but similarly do citations with journals in the cognitive
environment of the journal, particularly in the case of specialist journals.
Would one like to bring all these values to zero because they deviate from
the distributional expectation?

If one looks at the distribution of citations of a journal by other
journals, one typically finds the negative powerlaw distribution beyond a
certain number of citing/cited journals (Katz, 1999, 2000). When one plots
the powerlaw, however, a hook is visible. The first 10-100 journals follow
another distribution. This deviation from the powerlaw includes the journal
itself.

For those of you who receive this email in html, I provide the graph for six
journals below (using the Journal Citation Reports of the SCI 2003):

Figure 1: citation distribution (citing) of six journals over the full
journal set journals included in the JCR 2003.

This first part of the distribution is an interesting one, in my opinion,
because it informs us precisely about how the journal deviates from the
(expected) powerlaw distribution for reasons other than the distributional
expectations (Poisson, etc.). In other words, it informs us about a specific
structure in the data that operates upon the powerlaw distribution. This
selection mechanism can be appreciated as the cognitive structure of the
specialty or field. Thus, it should not be brought to zero by an a priori
normalization.

I agree that the main diagonal values seem to imply an effect beyond the
relevance of within-journal citations. Price (1981) and Noma (1982) have
proposed normalization procedures which do not reduce the main diagonal to
zero, but try to estimate another value for it. The expectation, however, is
that the additional effect is field-specific, that is, it may vary among
fields of science. Since we have no knowledge about these field-specific
effects, one should perhaps refrain from a priori normalization because one
may be throwing away the child with the bathwater. A reduction to zero for
the main diagonal, however, is in any case far too drastic unless one is
specifically interested in an impact factor of a journal as a black box. I
would then like to hear theoretical reasons why this measure might be
interesting to the research program.

With kind regards,

Loet

References:

Katz, J. S. (1999). The Self-Similar Science System. Research Policy, 28,
501-517.

Katz, J. S. (2000). Scale Independent Indicators and Research Evaluation.
Science & Public Policy, 27(1), 23-36.

Noma, E. (1982). An Improved Method for Analyzing Square Scientometric
Transaction Matrices,. Scientometrics 4, 297-316.

Price, D. J. de Solla. (1981). The Analysis of Square Matrices of
Scientometric Transactions,. Scientometrics 3, 55-63.

  _____

Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
loet at leydesdorff.net  <mailto:loet at leydesdorff.net> ;
http://www.leydesdorff.net/

The  <http://www.upublish.com/books/leydesdorff-sci.htm> Challenge of
Scientometrics ; The Self-Organization of
<http://www.upublish.com/books/leydesdorff.htm> the Knowledge-Based Society

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Ronald Rousseau
> Sent: Saturday, November 20, 2004 5:58 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: [SIGMETRICS] BMC Bioinformatics' delayed impact
>
> Dear Colleagues,
>
> Matthew J. Cockerill's editorial in BMC Bioinformatics [1]
> (brought to this List's attention by Eugene Garfield) is
> certainly to the point. Yet, I would like to make some remarks.
>
> The unofficial impact factor of BMC Bioinformatics as
> calculated by Cockerill is slightly different from those with
> which it is compared. Indeed, all other ones include journal
> self-citations while for BMC Bioinformatics, these journal
> self-citations are not included. On the other hand, when
> adding BMC Bioinformatics to the pool of source journals all
> other citation totals, and hence, impact factors possible
> change (if these other journals are cited by articles
> published by BMC Bioinformatics, and this, of course, within
> the time window considered).
>
> So, BMC Bioinformatics' impact factor is indeed an estimate.
>
> May I also stress the fact that this estimate is with respect
> to the 'ISI Universe'. There exist many journals, even
> important ones, outside this universe, as BMC Bioinformatics'
> editor knows very well. These journals are - by definition -
> not considered in ISI's  impact factor calculations.
>
> Finally our colleague Johannes Stegmann is given credit for
> this estimation method. Yet, to the best of my knowledge,
> Spaventi et al. [2] were the first to propose this approach
> (in 1979). It was later re-invented by B.K. Sen et al. [3].
> Johannes Stegmann certainly deserves credit for bringing this
> method to the attention of a wider audience [4], and, is,
> moreover, the person who made the most fundamental studies of
> the constructed impact factors (see e.g.
> [5]). Yet, BMC Bioinformatics' editor joins the crowd,
> conveniently, disregarding studies on impact factors
> published in information sciences journals.
>
>
> Ronald Rousseau
> 2004, November 20
>
>
> [1]. Cockerill, M.J. (2004). Delayed impact: ISI's  citation
> tracking choices are keeping scientists in the dark. BMC
> Bioinformatics, 5:93.
>
> [2]. Spaventi, J., Tudor-Silovic, N., Maricic, S. & Labus, M. (1979).
> Bibliometric analysis of scientific journals from Yugoslavia.
> Informatologia Yugoslavica, 11(3), 11-23.
>
> [3]. Sen, B.K., Karanjai, A., & Munshi, U.M. (1989). A method
> for determining the impact factor of a non-SCI journal.
> Journal of Documentation, 45(2), 139- 141.
>
> [4]. Stegmann, J. (1997). How to evaluate journal impact
> factors. Nature, 390, 550.
>
> [5]. Stegmann, J. (1999). Building a list of journals with
> constructed impact factors. Journal of Documentation, 55(3), 310-324.
>
> **********************************************************
> Ronald Rousseau
> KHBO - Industrial Sciences and Technology
> Zeedijk 101    B-8400  Oostende   Belgium
> Guest Professor at the Library of the Chinese Academy of
> Sciences Honorary Professor Henan Normal University (Xinxiang, China)
> E-mail: ronald.rousseau at khbo.be
> web page:  users.pandora.be/ronald.rousseau
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20041121/1694245f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: clip_image002.gif
Type: image/gif
Size: 6537 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20041121/1694245f/attachment.gif>