Impact Factor and ISI

Thu Sep 14 11:07:31 EDT 2000

ISI has been producing the JCR since 1975.  The 2000 edition, which will be
distributed next summer, will be its "silver anniversary" edition.  While
the product has developed over the years in terms of being offered in
different formats (print, fiche, CD, Web), its content has not.  Earlier
this year, I was given production responsibility for the JCR, and in that
role, one of my strategic goals is to create a plan for future development
of the product *content*.  I can, of course, make no promises about what
changes may be made, but it is my intention to survey the main groups of
users (bibliometric researchers, publishers, administrators, and librarians)
before making any recommendations.  We will most likely schedule a series of
group discussions next spring.  Until then, please rest assured that I will
continue to follow all the discussions on this list (and ISSI), although I
may not always be able to respond.

Helen Atkins

**************************************************
Helen Barsky Atkins
Director, Database Development
ISI
3501 Market Street
Philadelphia, PA  19104
215.386.0100  x1218
215.387.4706  (Fax)
helen.atkins at isinet.com
**************************************************

-----Original Message-----
From: Isidro F. Aguillo [mailto:isidro at CINDOC.CSIC.ES]
Sent: Wednesday, September 13, 2000 4:34 AM
To: SIGMETRICS at listserv.utk.edu
Subject: Re: [SIGMETRICS] Impact Factor in Germany and other
non-Englishcountries

PLEA FOR A NEW IMPACT FACTOR

This is very important topic for our discipline.

I do not think the primary problem is with the geographical coverage of
the Citation Indexes family and the scarce representation of non-english
speaking journals. **Obviously**, I would like to see a better coverage
of Third world countries and more spanish, french or german-language
journals, but as there is (now???): - evidence against the effect of
adding "second level" journals to global perfomance of the countries,
- the selection criteria for additional titles is matter of debate and,
- last, but not least, it is (very??) expensive to add many more
periodicals,
I suggest a different (more scientometric?? and commercial!!) approach.

Several authors pointed the problem is not with SCI and family of
databases, but with the rankinkgs of the JCR and specially with the
Impact Factor indicator. The indiscrimate use of IF, a fact every
scientometrician has warned about, is surely behind some of the biases
about "adequate" coverage of SCI+SSCI+AHCI.

During last decades several shortcommings of IF has been pointed and
alternative indexes has been suggested but ISI has never adopted any of
the suggested changes. As some of the proposals are mainly academic
exercises difficult to implement, I could understand ISI's attitude. But
there are, at least, several changes that are very EASY and ECONOMICAL
to make and they could improve significantly IF value.

The proposal is to generate annual JCR with a different set of indexes
including a new impact factor (NIF) with
- an expanded citation window (three or, better, four year period, not
more),
- excluding author self-citations and
- using a more restricted set of document types.
A second impact factor adjusting the values according discipline (CIF)
is becoming popular among scientometrics and it must be added. The other
statistics such as the immediacy or half-life indexes are seldom used (I
do not remember anybody using them ...).

There are several advantages for these new indexes. They are very easy
to calculate, so ISI do not need to change the way they compilate JCR.
Some other improvements could be added, but they are not "mandatory"
(mainly related with quality evaluation of the source database and the
classification of the journals by discipline).

During two or three years the annual JCR could provide both old and new
IFs, then only the new scheme. But it will be needed a product to cover
back years period with the new indexes. A **eighties+nineties combined
new fashion JCR** will be probably a "best seller" for ISI, so the
additional PROFIT they will obtain from it could help to change their
mind about this proposal.

For descriptive scientometrics this new product could open a "golden
age" as new analysis need to be done. Perhaps the empirical evidence
obtained could give new light to the country bias debate. An improved
(corrected??) JCR is good news also for managers, politicians and other
scientists. And it is a good COMMERCIAL opportunity for ISI!

Note: This proposal is made from compilation of ideas from several
authors. They publish even extensive empirical evidence and support. I
acknowledge them for every point but I am unable to make a complete
bibliography ;-).

> "Garfield, Eugene" wrote:
>
> It seems that no matter what ISI does its coverage of countries for
> which English is not the native language the word bias is used. Now
> comes the latest report from our friends at Leiden University in which
> it is claimed there is evidence of "serious language-bias in the use
> of citation analysis for the evaluation of national science systems.
> It seems that ISI covers too many German language publications. They
> report the "first evidence"  (sic!!) of such bias. In other words, if
> you remove the German language papers from the database you will
> increase the impact of Germany in the rankings.
>
> This monumental discovery is reported in " First evidence of serious
> language-bias in the use of citation analysis for the evaluation of
> national science systems." Research Evaluation v.9(2),p.155-6 (Aug.
> 2000). This provocative title is an implicit criticism of citation
> analysis, but in fact it is simply of way of saying-- do citation
> analysis our way.
>
> Regardless of the mischievous titling of the paper the main
> observation that must be made with respect to the use of any
> comprehensive literature database. The more inclusive it is of third
> world and non-English language publications, and therefore presumably
> better for information retrieval purposes, the lower will be the
> overall impact for the countries involved.
>
> They also suggest that there is a bias in computing the overall impact
> of US and UK output because there are no non-English journals.
> However, they fail to point out that because ISI serves the interests
> of librarians and scientist users there is a substantial coverage of
> low impact English language publications in all fields, many of which
> are in fact local as e.g. the Texas Journal of Science. Eugene
> Garfield
>
>
----------------------------------------------------------------------------
----
>
> Eugene Garfield, Ph.D.  E-mail: garfield at codex.cis.upenn.edu
>  Telephone:  (215)243-2205 // Fax: (215)387-1266
>  Web site: www.eugenegarfield.org
> President, American Society for Information Science & Technology
> (ASIS&T) -    www.asis.org
> Chairman Emeritus, ISI, 3501 Market St , Philadelphia, PA 19104-3389
>    www.isinet.com
> Pres.,Ed.-in-Chief,  The Scientist, 3600 Market St , Philadelphia, PA
> 19104-2645
>    www.the-scientist.com
>
> -----Original Message-----
> From: Michel J. Menou [mailto:Michel.Menou at WANADOO.FR]
> Sent: Sunday, September 10, 2000 10:31 AM
> To: SIGMETRICS at listserv.utk.edu
> Subject: Re: [SIGMETRICS] Impact Factor in Brazil
>
> Since not everybody on the list might be concerned with the use of
> citations and IF in research evaluation in countries and fields which
> do
> not belong to the big league, I'd  rather refer to the work of the
> late
> Mike Moravcsick and follow up efforts as reflected in the following
> publications:
>
> Moravcsik, M. J., (ed.).  "Strengthening the coverage of
> Third World Science. The bibliographic indicators of the Third
> World's contribution to science. Deliberations, conclusions and
> initiatives of an ad-hoc international task force for assessing the
> scientific output of the Third World. Eugene, OR., Institute of
> Theoretical Science, University of Oregon, 1986.
>
> Arvanitis, Rigas; Gaillard, Jacques, (eds.). Les indicateurs de
> science
> pour les pays en developpement. Science indicators for developing
> countries. Proceedings of the International conference on science
> indicators for developing countries;  Paris, 15-19 October 1990.
> Paris,
> ORSTOM Editions, 1992.
>
> The leap on the issue of imperialism simply referred to the fact that
> when
> a measure is built on the basis of the achievements, needs and choices
> of
> the more powerful, applying it indiscriminately to all others,
> including
> the less served, is an imposition.
> This is why the production of local resources is so important. A good
> example is SciELO (www.scielo.br)
>
> At 15:38 09/09/00 -0400, Gene Garfield wrote:
>
> >Here is the list of 17 journals from Brazil covered in the 1999
> Journal
> >Citation Reports. But I don't quite understand its relevance to the
> >comments that follow. In general, the more local low impact  journals
> that
> >are included in the calculation of impact, the lower will be the
> overall
> >impact of that country's impact. snip
> Sure. Thus Botswana will lag far behind Norway, even in Forestry. So
> what?
> Do they play in the same league? Do they have the same individual
> objectives and social role? If at least country rankings would be
> corrected
> on the basis of the number of active scientists, with some coefficient
> for
> their budgets.
>
> >snip
>
> Michel
>
> PS By the way, it is fair to mention that ISI supported part of the
> activities undertaken by the task force assembled by Mike Moravcsik.
> We
> even had a party at Gene's place.

--
************************************************************
Isidro F. AGUILLO                  isidro at cindoc.csic.es
------------------------------------------------------------
CINDOC-CSIC                        Tel: +34-91-563.54.82
Joaquin Costa, 22                  Fax: +34-91-564.26.44
28002 Madrid. ESPAÑA/SPAIN
Editor Cybermetrics (http://www.cindoc.csic.es/cybermetrics)
************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20000914/6b6b880a/attachment.html>