CACM, 50(9), (2007), p. 14

[Loet Leydesdorff]

Include Citations When Ranking Institutions and Scholars 
Communications of the ACM, Volume 50, Number 9 (2007), Pages 13-14 

In their article "Automatic and Versatile Publications Ranking for Research
Institutions and Scholars" (June 2007), Jie Ren and Richard N. Taylor showed
that automatic publication ranking can yield results similar to those from
manual ranking processes. Although they warned about the sensitivity of
measuring for parameter choice, they also suggested that reproducing the
rankings validates the use of the instrument for quality assessment.

In addition to numbers of publications, citations are also useful for
ranking. Of the 17 journals mentioned in the article, 10 were also in the
Science Citation Index-Expanded Version. A publication and citation count of
these 10 journals for the same period (1995-2003) yields completely
different rankings.

I've now extended the table in the article (see
<http://www.leydesdorff.net/Table1_CACM/index.htm>
users.fmg.uva.nl/leydesdorff/Table1_CACM/) to include publication and
citation rates for the top 50 computing graduate programs; the (Spearman)
correlation coefficients between the original rankings and the new ones are
on the order of 0.5. My rankings are based on the attribution of one full
point to an institution for each (co-authored) publication and its
corresponding citations. However, proportional attribution does not
significantly affect my results. Not only does the order change, but five of
the 50 institutions did not have a single publication attributed in the
Thompson ISI selection during the same eight years.

I don't claim that citation-based rankings are better than those published
previously. The reliability of bibliometric constructs and their validity as
indicators of quality are two different issues.

Loet Leydesdorff
Amsterdam, The Netherlands

  _____  

Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
 <mailto:loet at leydesdorff.net> loet at leydesdorff.net ;
<http://www.leydesdorff.net/> http://www.leydesdorff.net/ 

 
Now available:
<http://www.universal-publishers.com/book.php?method=ISBN&book=1581129378>
The Knowledge-Based Economy: Modeled, Measured, Simulated. 385 pp.; US$
18.95 
 <http://www.universal-publishers.com/book.php?method=ISBN&book=1581126956>
The Self-Organization of the Knowledge-Based Society;
<http://www.universal-publishers.com/book.php?method=ISBN&book=1581126816>
The Challenge of Scientometrics

 
 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20070903/d1ba42cb/attachment.html>