Science/NSF report

Loet Leydesdorff loet at LEYDESDORFF.NET
Sun Feb 17 02:51:35 EST 2008

Dear colleagues, 
I followed up on the discussion about the selection of a sample of journals
which was used for the NSF Science and Engineering Indicators 2008. I
distribute this email because of the public relevance of these indicators,
but anonymized the heading:
I studied the files of journals included in the analysis which you so kindly
sent me. The Science and Egineering Indicators 2008 mention on p. 5-37 (Vol.
1) that 4906 journals were included in the analysis in 2005 and you indicate
that the file with the filename JNL_FILE_Y2005_v3.XLS provided the base for
this statistics. 
This file contains 10,216 journal names in total; 5394 of these journals
contained publications in 2005 and 5393 match with the ISI database for this
same year using the ISSN. The ISI database (SCI + SSCI) contains 2132 other
journal names which were not included. The descriptive statistics are as
follows (in percentages): 

SCI+SSCI	 NSF2005	 % included	
journals	 7525	 5393	 71.67	
publications	 898892	 705971	 78.54	
citations	 23322568	 21631020	 92.75	
An analysis of variance between the group of journals covered by the NSF
(5393 journals) against the journals not-covered (2132) teaches that the
covered group has significantly different means on all relevant parameters: 

Thus, the smaller group scores higher on all these parameters. However, the
impact factor was not the criterion for the selection given that the
included group also contains journals with low impact:



impact factor












Std. Deviation










The question thus becomes urgent what the selection criteria are for these
5393 journals, and what justifies the further reduction to 4906 journals as
a basis for the NSF Indicators 2008. As we know, the various indicators are
sensitive to the choices of selection criteria. 


For example, part of the problem with "the decline of UK science" in the
1980s was that UK scientists tended to publish above average in journals
which were not included in the CHI journal set (for analytic reasons)
because these journals were relatively new journals.


Given the weight of the NSF report in the public debate and the transparency
required, perhaps you can further elucidate the choices made. 


With best wishes, 



Loet Leydesdorff



Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
 <mailto:loet at> loet at ;

Visiting Professor,  <> ISTIC,
Beijing; Honorary Fellow  <> SPRU, University
of Sussex 
Now available:
The Knowledge-Based Economy: Modeled, Measured, Simulated, 385 pp.; US$
The Self-Organization of the Knowledge-Based Society ;
The Challenge of Scientometrics

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: att5e616.gif
Type: image/gif
Size: 43851 bytes
Desc: not available
URL: <>

More information about the SIGMETRICS mailing list