Impact Factor in Germany and other non-English countries

[Johannes Stegmann]

The idea (and benefits) of an "Extra-JCR" listing cited-only journals
together with their "external" impact factors (constructed on the basis of
the cites received from ISI source journals) has been again mentioned on
this list very recently by Sinisa Maricic.

The problem ISI might face trying to calculate those impact factors are the
missing denominators (numbers of papers published) because the non-JCR
journals are also mostly non-ISI source journals. Thus, they had to
retrieve the missing data from other databases (as we do it).

But, because there is an increasing use of "evaluative bibliometrics"
(sometimes restricted only to impact factors by our "patrons" in
biomedicine), I guess we have some responsibility to find a way to include
non-JCR journals in such measures (not mentioning our very duty to warn
against a primitve use of bibliometic data - but these warnings are given
by ISI and practically everybody with a bit experience in this matter).

Using such additional impact factors of non-JCR journals will probably
result in lower average impact factors (for institutions or whole
countries). When I jumped into bibliometrics three years ago (really not
knowing what was waiting there) I did a short (and uncomplete) analysis on
some german medical faculties and found that some twenty percent of the
papers published by these faculties were published in journals not listed
in the JCR. These twenty percent gathered only two percent of the cites
(given by ISI source journals). Although the percentage would be higher if
all cites (including those given by the non-ISI journals) could be counted,
the fact will probably remain that the papers published in non-ISI journals
are, in general, cited only at a low rate.

Should we cut off these low cited papers (as it is implicitly done by ISI's
way) or should we live with lower averages? But where to cut? I think it
was mentioned in a paper by Seglen that even a short-cut tail ends up in a
tail which could be cut off.

Although I try to do some work on non-JCR journals I would argue that
bibliometric evaluation can be done (well) on the basis of ISI's tools
provided other means are included (e.g. peer review - provided the peers
don't rely only on bibliometrics).

Besides evaluative bibliometrics, citation analysis has other purposes
(e.g. "science mapping"), and this leads inevitably to the question "what
is science?" which was answered by Derek de Solla Price with the phrase
  "...to take as science that which is published in scientific papers".

Thus, from a scientific view, it is not so good that in citation analysis
all the references cited in non-ISI source journals cannot be included. One
has to count manually (which was (is?) actually done in some papers).
However, no one would have doubts that (at least in biomedicine) the really
relevant papers can be traced by ISI's citation facilities.
On the other hand, to argue against this statement, one would have to show
that there are "really relevant" papers published in journals not indexed
by ISI and not cited in ISI' source journals. I do not know such
investigations (I guess in a paper by Henry Small on AIDS it was mentioned
that one (ore more) highly cited papers were published in non-ISI journals,
but they were cited, of course, in ISI journals).

In another context, however, access to *all* cites would be desirable: in
order to find two sets of papers (in the sense of the work done by Don
Swanson) which are noninteractive, disjoint but complementary (containing
terms common to both sets), it is necessary to look wether paper(s) of one
set cite paper(s) of the other set, or wether papers of both sets are
co-cited. The published papers can be retrieved rather comprehensively from
the many databases available, but the citation and co-citation analysis is
restricted.

Johannes Stegmann


-------------------------------------------------------
Dr. Johannes Stegmann      Univ. Hospital Benjamin Franklin
Free University Berlin     Medical Library
stegmann at ukbf.fu-berlin.de Hindenburgdamm 30
Tel.: +49 30 8445 2035     D-12200 Berlin
Fax:  +49 30 8445 4454     Germany
  Homepage:  http://www.medizin.fu-berlin.de/medbib/home.html