Science/NSF report

[Loet Leydesdorff]

Dear colleagues, 
 
I followed up on the discussion about the selection of a sample of journals
which was used for the NSF Science and Engineering Indicators 2008. I
distribute this email because of the public relevance of these indicators,
but anonymized the heading:
 
I studied the files of journals included in the analysis which you so kindly
sent me. The Science and Egineering Indicators 2008 mention on p. 5-37 (Vol.
1) that 4906 journals were included in the analysis in 2005 and you indicate
that the file with the filename JNL_FILE_Y2005_v3.XLS provided the base for
this statistics. 
 
This file contains 10,216 journal names in total; 5394 of these journals
contained publications in 2005 and 5393 match with the ISI database for this
same year using the ISSN. The ISI database (SCI + SSCI) contains 2132 other
journal names which were not included. The descriptive statistics are as
follows (in percentages): 

 
 
SCI+SSCI	 NSF2005	 % included	
journals	 7525	 5393	 71.67	
publications	 898892	 705971	 78.54	
citations	 23322568	 21631020	 92.75	
 
An analysis of variance between the group of journals covered by the NSF
(5393 journals) against the journals not-covered (2132) teaches that the
covered group has significantly different means on all relevant parameters: 


Thus, the smaller group scores higher on all these parameters. However, the
impact factor was not the criterion for the selection given that the
included group also contains journals with low impact:

 

Statistics

impact factor



N

Valid

5372


Missing

21


Mean

1.84146


Median

1.13900


Mode

.500


Std. Deviation

2.752701


Range

49.794


Minimum

.000


Maximum

49.794

 

 

The question thus becomes urgent what the selection criteria are for these
5393 journals, and what justifies the further reduction to 4906 journals as
a basis for the NSF Indicators 2008. As we know, the various indicators are
sensitive to the choices of selection criteria. 

 

For example, part of the problem with "the decline of UK science" in the
1980s was that UK scientists tended to publish above average in journals
which were not included in the CHI journal set (for analytic reasons)
because these journals were relatively new journals.

 

Given the weight of the NSF report in the public debate and the transparency
required, perhaps you can further elucidate the choices made. 

 

With best wishes, 

 

 

Loet Leydesdorff

 

  _____  

Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
 <mailto:loet at leydesdorff.net> loet at leydesdorff.net ;
<http://www.leydesdorff.net/> http://www.leydesdorff.net/ 


Visiting Professor,  <http://www.istic.ac.cn/Eng/brief_en.html> ISTIC,
Beijing; Honorary Fellow  <http://www.sussex.ac.uk/spru/> SPRU, University
of Sussex 
Now available:
<http://www.universal-publishers.com/book.php?method=ISBN&book=1581129378>
The Knowledge-Based Economy: Modeled, Measured, Simulated, 385 pp.; US$
18.95;
 <http://www.universal-publishers.com/book.php?method=ISBN&book=1581126956>
The Self-Organization of the Knowledge-Based Society ;
<http://www.universal-publishers.com/book.php?method=ISBN&book=1581126816>
The Challenge of Scientometrics

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20080217/e466fca6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: att5e616.gif
Type: image/gif
Size: 43851 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20080217/e466fca6/attachment.gif>