Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Mon Mar 6 10:42:29 EST 2006

Hello everyone,

Regarding ISI subject categories, I don't know how they are picked, but
you can read about journal inclusion at
http://scientific.thomson.com/free/essays/selectionofmaterial/journalsel
ection/. I would assume that if this much effort is going into picking
journals to index, there is at least some level of thought going into
subject categorization. After all, the things ISI is looking at to
decide on journal inclusion are in many cases the same things you would
look at for categorization.

Now I won't claim that ISI's categorization is perfect, or that it is
updated as frequently as it might be, but I have yet to see anyone put
forth a better categorization scheme that is widely available. I'm sure
there are many of us that either have, or could easily, make up our own
schemes based on citation patterns or other factors, that we feel would
be more accurate.

Now I'll go out on a limb with an opinion that is counter to what I see
in many papers. On the whole I think the ISI categories are pretty good.
There are some categories that should be thrown out, some that should be
split, some that could be merged, and some journals that are clearly
mis-categorized, as mentioned in my recent Scientometrics [v64(3),
p351-374] paper with Klavans and Borner. We can have lots of discussion
about the details, but when you look at the whole thing, it's a good
start.

Perhaps I'm more comfortable than most with the ISI structure because I
don't rely on it for evaluation. We use the emergent cluster structure
from papers rather than any category or journal scheme. But that's
another story.

Can anyone recommend a better journal categorization scheme that is
widely available? I'd love to hear of (and get a copy of) such a thing.

Best regards,
Kevin

-----Original Message-----
From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
Sent: Saturday, March 04, 2006 6:15 AM
To: SIGMETRICS at listserv.utk.edu
Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Dear colleagues,

The idea is interesting. However, there a few problems with this paper.
First, the authors should not have used Pearson correlation coefficients
to compare the rankings, but rank correlations (Spearman's rho or
Kendall's tau). Second, it would have been interesting to have a rank
correlation with "total cites" given recent discussions (Bensman).
Third, the delineation of fields in terms of the ISI subject categories
is very questionnable.

However, the authors are very clear about their results: "We identified
...
, but were unable to recognize a meaningful pattern in the results." (p.
9).
I don't understand why one should then multiply the one measure with the
other. What does multiplication to the error?

Does one of you know a place where the ISI subject categories are
justified?
How are they produced? People seem to use them increasingly both in
evaluation and research practices, but I have never been able to
reproduce them using journal citation measures.

With best wishes,

Loet

________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR), Kloveniersburgwal
48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681; loet at leydesdorff.net ;
http://www.leydesdorff.net/

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Eugene Garfield
> Sent: Friday, March 03, 2006 6:37 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
> Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1
> 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Further to yesterday's posting, "Prestige is factored into journal
> ratings", here is another interesting and informative article
>
> FULL TEXT AVAILABLE AT :
> http://www.arxiv.org/PS_cache/cs/pdf/0601/0601030.pdf
>
> email: {jbollen, marko, herbertv}@lanl.gov
>
> TITLE   :  Journal Status
>
> AUTHORS :  Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel
>
> SOURCE   : arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Abstract
> The status of an actor in a social context is commonly defined in
> terms of two factors: the total number of endorsements the actor
> receives from other actors and the prestige of the endorsing actors.
> These two factors indicate the distinction between popularity and
> expert appreciation of the actor, respectively. We refer to the former

> as popularity and to the latter as prestige. These notions of
> popularity and prestige also apply to the domain of scholarly
> assessment. The ISI Impact Factor (ISI IF) is defined as the mean
> number of citations a journal receives over a 2 year period. By merely

> counting the amount of citations and disregarding the prestige of the
> citing journals, the ISI IF is a metric of popularity, not of
> prestige. We demonstrate how a weighted version of the popular
> PageRank algorithm can be used to obtain a metric that reflects
> prestige. We contrast the rankings of journals according to their ISI
> IF and their weighted PageRank, and we provide an analysis that
> reveals both significant overlaps and differences.
> Furthermore, we introduce the Y-factor which is a simple combination
> of both the ISI IF and the weighted PageRank, and find that the
> resulting journal rankings correspond well to a general understanding
> of journal status.
>
>
> ______________________________________________
>
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> FULL TEXT AVAILABLE AT :
> http://www.nature.com/nature/journal/v439/n7078/pdf/439770a.pdf    OR
> http://guide.labanimal.com/news/2006/060213/full/439770a.html
>
>
> Philip Ball : p.ball at nature.com
> www.philipball.com
>
> Title: Prestige is factored into journal ratings
>
> Author(s): Ball P
>
> Source: NATURE 439 (7078): 770-771 FEB 16 2006
>
> Document Type: News Item Language: English
> Cited References: 0      Times Cited: 0
>
> Publisher: NATURE PUBLISHING GROUP, MACMILLAN BUILDING, 4 CRINAN ST,
> LONDON
> N1 9XW, ENGLAND
> Subject Category: MULTIDISCIPLINARY SCIENCES IDS Number: 012JA
>
> ISSN: 0028-0836
>