Impact Factor and ISI

Dr. John V. Richardson Jr. jrichard at UCLA.EDU
Thu Sep 14 11:59:04 EDT 2000

[SIGMETRICS] Impact Factor and ISIHelen, I think ISI's JCR plays a valuable role in many scientific fields.  Having followed it as editor of LQ, I have noticed certain things about it that made me passingly curious, but recently I started a study of leading as well as international LIS journals.

I would really like to see operational definitions for how articles are counted because I have found inconsistencies in JCR.  Here is what I finally decided to do; perhaps it will help you and others in their own studies:

1)      Total number of major (main, full-length) articles per year (based on typography):

a)      No annotated, classified (or otherwise) bibliographies, notable documents or reviews of reference sources

b)     No editorials or guest editorials, symposia, research in brief, critical reviews, in memoriam (even with citations or references), introductions to special issues, brief or shorter communications, columns, conference reports, critical review articles, notes, opinion pieces, comments or letters or responses, etc.

c)      Book reviews are not counted

d)     Count "Interviews" when accompanied with notes or references

Best wishes, JRjr


  ----- Original Message ----- 
  From: Atkins, Helen 
  Sent: Thursday, September 14, 2000 8:07 AM
  Subject: [SIGMETRICS] Impact Factor and ISI

  ISI has been producing the JCR since 1975.  The 2000 edition, which will be distributed next summer, will be its "silver anniversary" edition.  While the product has developed over the years in terms of being offered in different formats (print, fiche, CD, Web), its content has not.  Earlier this year, I was given production responsibility for the JCR, and in that role, one of my strategic goals is to create a plan for future development of the product *content*.  I can, of course, make no promises about what changes may be made, but it is my intention to survey the main groups of users (bibliometric researchers, publishers, administrators, and librarians) before making any recommendations.  We will most likely schedule a series of group discussions next spring.  Until then, please rest assured that I will continue to follow all the discussions on this list (and ISSI), although I may not always be able to respond.

  Helen Atkins 

  Helen Barsky Atkins 
  Director, Database Development 
  3501 Market Street 
  Philadelphia, PA  19104 
  215.386.0100  x1218 
  215.387.4706  (Fax) 
  helen.atkins at 

  -----Original Message----- 
  From: Isidro F. Aguillo [mailto:isidro at CINDOC.CSIC.ES] 
  Sent: Wednesday, September 13, 2000 4:34 AM 
  Subject: Re: [SIGMETRICS] Impact Factor in Germany and other 


  This is very important topic for our discipline. 

  I do not think the primary problem is with the geographical coverage of 
  the Citation Indexes family and the scarce representation of non-english 
  speaking journals. **Obviously**, I would like to see a better coverage 
  of Third world countries and more spanish, french or german-language 
  journals, but as there is (now???): - evidence against the effect of 
  adding "second level" journals to global perfomance of the countries, 
  - the selection criteria for additional titles is matter of debate and, 
  - last, but not least, it is (very??) expensive to add many more 
  I suggest a different (more scientometric?? and commercial!!) approach. 

  Several authors pointed the problem is not with SCI and family of 
  databases, but with the rankinkgs of the JCR and specially with the 
  Impact Factor indicator. The indiscrimate use of IF, a fact every 
  scientometrician has warned about, is surely behind some of the biases 
  about "adequate" coverage of SCI+SSCI+AHCI. 

  During last decades several shortcommings of IF has been pointed and 
  alternative indexes has been suggested but ISI has never adopted any of 
  the suggested changes. As some of the proposals are mainly academic 
  exercises difficult to implement, I could understand ISI's attitude. But 
  there are, at least, several changes that are very EASY and ECONOMICAL 
  to make and they could improve significantly IF value. 

  The proposal is to generate annual JCR with a different set of indexes 
  including a new impact factor (NIF) with 
  - an expanded citation window (three or, better, four year period, not 
  - excluding author self-citations and 
  - using a more restricted set of document types. 
  A second impact factor adjusting the values according discipline (CIF) 
  is becoming popular among scientometrics and it must be added. The other 
  statistics such as the immediacy or half-life indexes are seldom used (I 
  do not remember anybody using them ...). 

  There are several advantages for these new indexes. They are very easy 
  to calculate, so ISI do not need to change the way they compilate JCR. 
  Some other improvements could be added, but they are not "mandatory" 
  (mainly related with quality evaluation of the source database and the 
  classification of the journals by discipline). 

  During two or three years the annual JCR could provide both old and new 
  IFs, then only the new scheme. But it will be needed a product to cover 
  back years period with the new indexes. A **eighties+nineties combined 
  new fashion JCR** will be probably a "best seller" for ISI, so the 
  additional PROFIT they will obtain from it could help to change their 
  mind about this proposal. 

  For descriptive scientometrics this new product could open a "golden 
  age" as new analysis need to be done. Perhaps the empirical evidence 
  obtained could give new light to the country bias debate. An improved 
  (corrected??) JCR is good news also for managers, politicians and other 
  scientists. And it is a good COMMERCIAL opportunity for ISI! 

  Note: This proposal is made from compilation of ideas from several 
  authors. They publish even extensive empirical evidence and support. I 
  acknowledge them for every point but I am unable to make a complete 
  bibliography ;-). 

  > "Garfield, Eugene" wrote: 
  > It seems that no matter what ISI does its coverage of countries for 
  > which English is not the native language the word bias is used. Now 
  > comes the latest report from our friends at Leiden University in which 
  > it is claimed there is evidence of "serious language-bias in the use 
  > of citation analysis for the evaluation of national science systems. 
  > It seems that ISI covers too many German language publications. They 
  > report the "first evidence"  (sic!!) of such bias. In other words, if 
  > you remove the German language papers from the database you will 
  > increase the impact of Germany in the rankings. 
  > This monumental discovery is reported in " First evidence of serious 
  > language-bias in the use of citation analysis for the evaluation of 
  > national science systems." Research Evaluation v.9(2),p.155-6 (Aug. 
  > 2000). This provocative title is an implicit criticism of citation 
  > analysis, but in fact it is simply of way of saying-- do citation 
  > analysis our way. 
  > Regardless of the mischievous titling of the paper the main 
  > observation that must be made with respect to the use of any 
  > comprehensive literature database. The more inclusive it is of third 
  > world and non-English language publications, and therefore presumably 
  > better for information retrieval purposes, the lower will be the 
  > overall impact for the countries involved. 
  > They also suggest that there is a bias in computing the overall impact 
  > of US and UK output because there are no non-English journals. 
  > However, they fail to point out that because ISI serves the interests 
  > of librarians and scientist users there is a substantial coverage of 
  > low impact English language publications in all fields, many of which 
  > are in fact local as e.g. the Texas Journal of Science. Eugene 
  > Garfield 
  > -------------------------------------------------------------------------------- 
  > Eugene Garfield, Ph.D.  E-mail: garfield at 
  >  Telephone:  (215)243-2205 // Fax: (215)387-1266 
  >  Web site: 
  > President, American Society for Information Science & Technology 
  > (ASIS&T) - 
  > Chairman Emeritus, ISI, 3501 Market St , Philadelphia, PA 19104-3389 
  > Pres.,Ed.-in-Chief,  The Scientist, 3600 Market St , Philadelphia, PA 
  > 19104-2645 
  > -----Original Message----- 
  > From: Michel J. Menou [mailto:Michel.Menou at WANADOO.FR] 
  > Sent: Sunday, September 10, 2000 10:31 AM 
  > To: SIGMETRICS at 
  > Subject: Re: [SIGMETRICS] Impact Factor in Brazil 
  > Since not everybody on the list might be concerned with the use of 
  > citations and IF in research evaluation in countries and fields which 
  > do 
  > not belong to the big league, I'd  rather refer to the work of the 
  > late 
  > Mike Moravcsick and follow up efforts as reflected in the following 
  > publications: 
  > Moravcsik, M. J., (ed.).  "Strengthening the coverage of 
  > Third World Science. The bibliographic indicators of the Third 
  > World's contribution to science. Deliberations, conclusions and 
  > initiatives of an ad-hoc international task force for assessing the 
  > scientific output of the Third World. Eugene, OR., Institute of 
  > Theoretical Science, University of Oregon, 1986. 
  > Arvanitis, Rigas; Gaillard, Jacques, (eds.). Les indicateurs de 
  > science 
  > pour les pays en developpement. Science indicators for developing 
  > countries. Proceedings of the International conference on science 
  > indicators for developing countries;  Paris, 15-19 October 1990. 
  > Paris, 
  > ORSTOM Editions, 1992. 
  > The leap on the issue of imperialism simply referred to the fact that 
  > when 
  > a measure is built on the basis of the achievements, needs and choices 
  > of 
  > the more powerful, applying it indiscriminately to all others, 
  > including 
  > the less served, is an imposition. 
  > This is why the production of local resources is so important. A good 
  > example is SciELO ( 
  > At 15:38 09/09/00 -0400, Gene Garfield wrote: 
  > >Here is the list of 17 journals from Brazil covered in the 1999 
  > Journal 
  > >Citation Reports. But I don't quite understand its relevance to the 
  > >comments that follow. In general, the more local low impact  journals 
  > that 
  > >are included in the calculation of impact, the lower will be the 
  > overall 
  > >impact of that country's impact. snip 
  > Sure. Thus Botswana will lag far behind Norway, even in Forestry. So 
  > what? 
  > Do they play in the same league? Do they have the same individual 
  > objectives and social role? If at least country rankings would be 
  > corrected 
  > on the basis of the number of active scientists, with some coefficient 
  > for 
  > their budgets. 
  > >snip 
  > Michel 
  > PS By the way, it is fair to mention that ISI supported part of the 
  > activities undertaken by the task force assembled by Mike Moravcsik. 
  > We 
  > even had a party at Gene's place. 

  Isidro F. AGUILLO                  isidro at 
  CINDOC-CSIC                        Tel: +34-91-563.54.82 
  Joaquin Costa, 22                  Fax: +34-91-564.26.44 
  28002 Madrid. ESPAÑA/SPAIN 
  Editor Cybermetrics ( 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the SIGMETRICS mailing list