Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Mon Mar 6 14:53:48 EST 2006

Dear Kevin,

The above figure is from my paper entitled
<http://www.leydesdorff.net/classif03/index.htm> Can Scientific Journals be
Classified in terms of Aggregated Journal-Journal Citation Relations using
the Journal Citation Reports? Journal of the American Society for
Information Science and Technology (forthcoming). <
<http://www.leydesdorff.net/classif03/classif03.pdf> pdf-version> . It made
me less optimistic than you about the quality of the classification and I
asked therefore one colleague at ISI how this classification was made, and
he answered that he thought that there was no serious analytical basis for
it. My question at this list was more to ask whether someone knew the basis
of it than for being reassured.

It seems very worrysome to me that this classification is increasingly used
while it seems based on ....? I cannot find any confirmation. Your inference
that the discussion about inclusion warrants the quality of the
classification is also not warranted by research.

With best wishes,

Loet

________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
loet at leydesdorff.net ; http://www.leydesdorff.net/

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Boyack, Kevin W
> Sent: Monday, March 06, 2006 4:42 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez,
> and Herbert Van de Sompel "Journal Status"
> arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Hello everyone,
>
> Regarding ISI subject categories, I don't know how they are
> picked, but you can read about journal inclusion at
> http://scientific.thomson.com/free/essays/selectionofmaterial/
> journalsel
> ection/. I would assume that if this much effort is going
> into picking journals to index, there is at least some level
> of thought going into subject categorization. After all, the
> things ISI is looking at to decide on journal inclusion are
> in many cases the same things you would look at for categorization.
>
> Now I won't claim that ISI's categorization is perfect, or
> that it is updated as frequently as it might be, but I have
> yet to see anyone put forth a better categorization scheme
> that is widely available. I'm sure there are many of us that
> either have, or could easily, make up our own schemes based
> on citation patterns or other factors, that we feel would be
> more accurate.
>
> Now I'll go out on a limb with an opinion that is counter to
> what I see in many papers. On the whole I think the ISI
> categories are pretty good.
> There are some categories that should be thrown out, some
> that should be split, some that could be merged, and some
> journals that are clearly mis-categorized, as mentioned in my
> recent Scientometrics [v64(3), p351-374] paper with Klavans
> and Borner. We can have lots of discussion about the details,
> but when you look at the whole thing, it's a good start.
>
> Perhaps I'm more comfortable than most with the ISI structure
> because I don't rely on it for evaluation. We use the
> emergent cluster structure from papers rather than any
> category or journal scheme. But that's another story.
>
> Can anyone recommend a better journal categorization scheme
> that is widely available? I'd love to hear of (and get a copy
> of) such a thing.
>
> Best regards,
> Kevin
>
>
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
> Sent: Saturday, March 04, 2006 6:15 AM
> To: SIGMETRICS at listserv.utk.edu
> Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez,
> and Herbert Van de Sompel "Journal Status"
> arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Dear colleagues,
>
> The idea is interesting. However, there a few problems with
> this paper.
> First, the authors should not have used Pearson correlation
> coefficients to compare the rankings, but rank correlations
> (Spearman's rho or Kendall's tau). Second, it would have been
> interesting to have a rank correlation with "total cites"
> given recent discussions (Bensman).
> Third, the delineation of fields in terms of the ISI subject
> categories is very questionnable.
>
> However, the authors are very clear about their results: "We
> identified ...
> , but were unable to recognize a meaningful pattern in the
> results." (p.
> 9).
> I don't understand why one should then multiply the one
> measure with the other. What does multiplication to the error?
>
> Does one of you know a place where the ISI subject categories
> are justified?
> How are they produced? People seem to use them increasingly
> both in evaluation and research practices, but I have never
> been able to reproduce them using journal citation measures.
>
> With best wishes,
>
>
> Loet
>
>
> ________________________________
> Loet Leydesdorff
> Amsterdam School of Communications Research (ASCoR),
> Kloveniersburgwal 48, 1012 CX Amsterdam.
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
> loet at leydesdorff.net ; http://www.leydesdorff.net/
>
>
>
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Eugene Garfield
> > Sent: Friday, March 03, 2006 6:37 PM
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
> > Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1
> > 9 Jan 2006
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Further to yesterday's posting, "Prestige is factored into journal
> > ratings", here is another interesting and informative article
> >
> > FULL TEXT AVAILABLE AT :
> > http://www.arxiv.org/PS_cache/cs/pdf/0601/0601030.pdf
> >
> > email: {jbollen, marko, herbertv}@lanl.gov
> >
> > TITLE   :  Journal Status
> >
> > AUTHORS :  Johan Bollen, Marko A. Rodriguez, and Herbert
> Van de Sompel
> >
> > SOURCE   : arXiv:cs.GL/0601030 v1 9 Jan 2006
> >
> > Abstract
> > The status of an actor in a social context is commonly defined in
> > terms of two factors: the total number of endorsements the actor
> > receives from other actors and the prestige of the endorsing actors.
> > These two factors indicate the distinction between popularity and
> > expert appreciation of the actor, respectively. We refer to
> the former
>
> > as popularity and to the latter as prestige. These notions of
> > popularity and prestige also apply to the domain of scholarly
> > assessment. The ISI Impact Factor (ISI IF) is defined as the mean
> > number of citations a journal receives over a 2 year
> period. By merely
>
> > counting the amount of citations and disregarding the
> prestige of the
> > citing journals, the ISI IF is a metric of popularity, not of
> > prestige. We demonstrate how a weighted version of the popular
> > PageRank algorithm can be used to obtain a metric that reflects
> > prestige. We contrast the rankings of journals according to
> their ISI
> > IF and their weighted PageRank, and we provide an analysis that
> > reveals both significant overlaps and differences.
> > Furthermore, we introduce the Y-factor which is a simple
> combination
> > of both the ISI IF and the weighted PageRank, and find that the
> > resulting journal rankings correspond well to a general
> understanding
> > of journal status.
> >
> >
> > ______________________________________________
> >
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > FULL TEXT AVAILABLE AT :
> >
> http://www.nature.com/nature/journal/v439/n7078/pdf/439770a.pdf    OR
> > http://guide.labanimal.com/news/2006/060213/full/439770a.html
> >
> >
> > Philip Ball : p.ball at nature.com
> > www.philipball.com
> >
> > Title: Prestige is factored into journal ratings
> >
> > Author(s): Ball P
> >
> > Source: NATURE 439 (7078): 770-771 FEB 16 2006
> >
> > Document Type: News Item Language: English
> > Cited References: 0      Times Cited: 0
> >
> > Publisher: NATURE PUBLISHING GROUP, MACMILLAN BUILDING, 4
> CRINAN ST,
> > LONDON
> > N1 9XW, ENGLAND
> > Subject Category: MULTIDISCIPLINARY SCIENCES IDS Number: 012JA
> >
> > ISSN: 0028-0836
> >
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060306/50d92a58/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fig12.jpg
Type: image/jpeg
Size: 36597 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060306/50d92a58/attachment.jpg>