Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

[Loet Leydesdorff]

Dear Stephen,

In your categorization, I am fullheartedly on the American side. I would be
very hesitant to use the word quality in this context. My interest is in
scientific communication, its structures, and its development.

With best wishes,

Loet
----- Original Message -----
From: "Stephen J Bensman" <notsjb at LSU.EDU>
To: <SIGMETRICS at LISTSERV.UTK.EDU>
Sent: Thursday, March 09, 2006 3:28 PM
Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert Van
de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006


> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Loet,
> That is pretty basic, but the problem is that due to Bradford's Law and
> Garfield's Law definition of such sets is impossible.  You are always
> going
> to have exogenous variables if you use citations.  If you going to use
> citations in evaluations, they must be used together with other
> variables--the best being experta ratings if such are available.  Then you
> can check for extreme outliers indicating sources of distortion.  The
> evaluation must be specific to those scientists  being evaluated.  It is
> not possible define mathematically sets universally applicable.
>
> The one thing that really bothers me about European research is that they
> seem to assume that citations are valid measures of quality.  It then
> concentrates on find some mathematical technique supposedly capable of
> measuring quality.  This research seems woefully short of studies of the
> opinions of actual scientists as well as the institutional and social
> bases
> of citations.  It seems to boil down to fascination with new
> gimmickry--latest being the present fad with the Hirsch index,  Compared
> with the work done by the American Council on Education and the US
> National
> Research Council it is quite crude--even the vaunted British RAE.  What
> the
> Americans have found is that no matter how carefully you do it, you always
> crap it up somehow.
>
> SB
>
>
>
>
> Loet Leydesdorff <loet at LEYDESDORFF.NET>@LISTSERV.UTK.EDU> on 03/09/2006
> 12:25:45 AM
>
> Please respond to ASIS&T Special Interest Group on Metrics
>       <SIGMETRICS at LISTSERV.UTK.EDU>
>
> Sent by:    ASIS&T Special Interest Group on Metrics
>       <SIGMETRICS at LISTSERV.UTK.EDU>
>
>
> To:    SIGMETRICS at LISTSERV.UTK.EDU
> cc:     (bcc: Stephen J Bensman/notsjb/LSU)
>
> Subject:    Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
>       Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
>> What's not to laugh about it?  There probably is no better
>> way to do it.
>>
>> SB
>
> Let me then repeat the problem: comparisons in terms of impact factors,
> etc., are valid only within cognitive domains with common citation and
> publication practices. In other words, citation graphs among journals have
> different densities and this affects the impact factors in the
> corresponding
> domains. For example, impact factors of immunology journals are much
> higher
> than impact factors of toxicology journals.
>
> The delineation of the sets in which one can compare thus matters. We know
> that this delineation cannot be perfect, but it matters how good they are.
> Increasingly evaluation commission and scientometric researchers seem to
> assume that the ISI subject categories are valid delineation of domains
> within which one can make comparisons. The article by Bollen et al. was a
> point in case.
>
> With best wishes,
>
>
> Loet
> ________________________________
> Loet Leydesdorff
> Amsterdam School of Communications Research (ASCoR),
> Kloveniersburgwal 48, 1012 CX Amsterdam.
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
> loet at leydesdorff.net ; http://www.leydesdorff.net/