Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance

[Francis Narin]

Dear Ulf,

    Loet's recollection is correct. I remember the car ride quite clearly. In addition I did interact a lot with Van Raan and others in the group at Leiden in the 1980's and 1990's , and also with the groups at SPRU, and in London, Paris, Bielefeld and elsewhere in Europe.

        Francis Narin
  ----- Original Message ----- 
  From: Loet Leydesdorff 
  To: SIGMETRICS at LISTSERV.UTK.EDU 
  Sent: Friday, April 09, 2010 12:03 PM
  Subject: Re: [SIGMETRICS] Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance


  Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 
    It was published just before the groups in Leiden and Leuven came with their first articles on normalized citation analysis, but had quite a different, and methodologically sound approach. In 2006, I visited Barry Bozeman in the US and gave a presentation on field normalized citation analysis. Immediately, he responded in a way that pointed to z-score as the best alternative for handling these types of problems. Could it be the case that in the 1980s there was a gap between the US and Europe, maybe there still is, regarding statistical methods and methdological competence? 
  Dear Ulf, 

  Thank you for briging this paper to our attention. In the early 1980s there was a lifely relation between the US and European scholars in bibliometrics. We met, for example, at the meetings of the 4S (The Society for the Social Studies of Science). I remember, for example, a car ride from a meeting in Troy (NY) with Fran Narin as the driver and Henk Moed and me in the car. Fran was going to show us his company in Philadelphia, but I felt not well and stepped out of the car in Newark. 

  Thus, we knew one another quite well. And certainly, we all read the same journals. (I must confess that I had never read this excellent piece).

  Best wishes, 


  Loet

    Since 2005, my group have performed citation analysis with normalization at article level, and in 2007 we included the Standard Citation Score using the method proposed by Mcallister et al. Most or our work is in Swedish, but to give a couple of examples (in English) of this type of work: see the report on the Research Assessement in 2008 for Royal Institute of Technology, Stockholm, avaiable here: http://www.forskningspolitik.se/DataFile.asp?FileID=165 or via www.se/rae. Another evaluation using the same methods was performed last year for the Swedish Environmental Protection Agency "Bibliometric evaluation of research programs": http://www.forskningspolitik.se/DataFile.asp?FileID=182.  Especially, have a look at the Technical Appendices to these reports.

    Best,
    Ulf Sandstrom


    Ulf Sandström, docent 


    Linköpings universitet                 

    ISAK                                        

    581 83 Linköping                        

    (           +46 708 137376
    *           ulf.sandstrom at liu.se 
    "     www.forskningspolitik.se  





----------------------------------------------------------------------------
    Från: Loet Leydesdorff [mailto:loet at LEYDESDORFF.NET] 
    Skickat: den 23 mars 2010 07:41
    Ämne: Re: Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance


    Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 
    Normalization, CWTS indicators, and the Leiden Rankings: 
    Differences in citation behavior at the level of fields
    Authors: Loet Leydesdorff, Tobias Opthof
    (Submitted on 21 Mar 2010)
      Abstract: Van Raan et al. (2010; arXiv:1003.2113) have proposed a new indicator (MNCS) for field normalization. Since field normalization is also used in the Leiden Rankings of universities, we elaborate our critique of journal normalization in Opthof & Leydesdorff (2010; arXiv:1002.2769) in this rejoinder concerning field normalization. Fractional citation counting thoroughly solves the issue of normalization for differences in citation behavior among fields. This indicator can also be used to obtain a normalized impact factor. 
          Subjects:  Physics and Society (physics.soc-ph) 
          Cite as:  arXiv:1003.3977v1 [physics.soc-ph] 



----------------------------------------------------------------------------

    Loet Leydesdorff 
    Amsterdam School of Communications Research (ASCoR)
    Kloveniersburgwal 48, 1012 CX Amsterdam
    loet at leydesdorff.net ; http://www.leydesdorff.net/ 




------------------------------------------------------------------------------



  No virus found in this incoming message.
  Checked by AVG - www.avg.com 
  Version: 9.0.801 / Virus Database: 271.1.1/2800 - Release Date: 04/08/10 23:32:00
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20100409/1aacecba/attachment.html>