Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Mon Mar 6 17:01:23 EST 2006

Dear Loet,

Your choice of category to highlight provides for some very interesting
points. "Biochemistry and Molecular Biology" is a very large and diverse
grouping in the ISI data. Journals are jointly categorized in this
category with some 59 other categories, which fact by itself says that
this category is too broad - thus supporting your argument. The largest
multiple categorizations are (2002 data) with:

CELL BIOLOGY    71
BIOPHYSICS    45
BIOTECH & APPLIED MICRO    24
GENETICS & HEREDITY    23
PHARMACOLOGY & PHARMACY    23
BIOCHEM RESEARCH METHODS    16
ENDOCRINOLOGY & METABOLISM    15
PLANT SCIENCES    13
IMMUNOLOGY    11
and so on.

But this also says to me that your comparison below is showing a bit of
"apples and oranges". For instance, one would not expect journals that
are jointly categorized BIOCHEM / BIOPHYSICS or PHARMA or PLANT SCIENCES
... to appear in the overlap set of your venn diagrams. Your factor
analysis categorizations would have put these journals in factors
different than those you highlight below (JACS-related and JBC-related),
and it is not entirely fair to include them. If you were to limit your
ISI comparison set to journals either singly categorized as BIOCHEM or
to those plus journals doubly categorized to BIOCHEM + CELL BIOLOGY,
your pictures would look much different. I would love to see that
analysis. When I look at the distribution of the ISI categories in my
cluster maps, and see that, for the most part, 80% of the members of
each cluster belong to a dominant ISI category, I continue to retain my
optimism. Regarding how categories are assigned - I am perhaps too much
of an optimist, but I have the hope that whoever is doing it is smart,
and is relying on some data to make judgments.

You are right about the pitfalls of using this categorization blindly -
I share your concerns. But people will increasingly use it until
something is shown to be better, and becomes readily available and
publicly accepted. Your local citation environments are a great step
forward. I'd love to see them in more common use.

Best regards,
Kevin

________________________________

From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
Sent: Monday, March 06, 2006 12:54 PM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Dear Kevin, 

The above figure is from my paper entitled Can Scientific Journals be
Classified in terms of Aggregated Journal-Journal Citation Relations
using the Journal Citation Reports?
<http://www.leydesdorff.net/classif03/index.htm>  Journal of the
American Society for Information Science and Technology (forthcoming). <
pdf-version <http://www.leydesdorff.net/classif03/classif03.pdf> > . It
made me less optimistic than you about the quality of the classification
and I asked therefore one colleague at ISI how this classification was
made, and he answered that he thought that there was no serious
analytical basis for it. My question at this list was more to ask
whether someone knew the basis of it than for being reassured.

It seems very worrysome to me that this classification is increasingly
used while it seems based on ....? I cannot find any confirmation. Your
inference that the discussion about inclusion warrants the quality of
the classification is also not warranted by research. 

With best wishes, 

Loet

________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
loet at leydesdorff.net ; http://www.leydesdorff.net/

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Boyack, Kevin W
> Sent: Monday, March 06, 2006 4:42 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez,
> and Herbert Van de Sompel "Journal Status"
> arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Hello everyone,
>
> Regarding ISI subject categories, I don't know how they are
> picked, but you can read about journal inclusion at
> http://scientific.thomson.com/free/essays/selectionofmaterial/
> journalsel
> ection/. I would assume that if this much effort is going
> into picking journals to index, there is at least some level
> of thought going into subject categorization. After all, the
> things ISI is looking at to decide on journal inclusion are
> in many cases the same things you would look at for categorization.
>
> Now I won't claim that ISI's categorization is perfect, or
> that it is updated as frequently as it might be, but I have
> yet to see anyone put forth a better categorization scheme
> that is widely available. I'm sure there are many of us that
> either have, or could easily, make up our own schemes based
> on citation patterns or other factors, that we feel would be
> more accurate.
>
> Now I'll go out on a limb with an opinion that is counter to
> what I see in many papers. On the whole I think the ISI
> categories are pretty good.
> There are some categories that should be thrown out, some
> that should be split, some that could be merged, and some
> journals that are clearly mis-categorized, as mentioned in my
> recent Scientometrics [v64(3), p351-374] paper with Klavans
> and Borner. We can have lots of discussion about the details,
> but when you look at the whole thing, it's a good start.
>
> Perhaps I'm more comfortable than most with the ISI structure
> because I don't rely on it for evaluation. We use the
> emergent cluster structure from papers rather than any
> category or journal scheme. But that's another story.
>
> Can anyone recommend a better journal categorization scheme
> that is widely available? I'd love to hear of (and get a copy
> of) such a thing.
>
> Best regards,
> Kevin
>
>
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
> Sent: Saturday, March 04, 2006 6:15 AM
> To: SIGMETRICS at listserv.utk.edu
> Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez,
> and Herbert Van de Sompel "Journal Status"
> arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Dear colleagues,
>
> The idea is interesting. However, there a few problems with
> this paper.
> First, the authors should not have used Pearson correlation
> coefficients to compare the rankings, but rank correlations
> (Spearman's rho or Kendall's tau). Second, it would have been
> interesting to have a rank correlation with "total cites"
> given recent discussions (Bensman).
> Third, the delineation of fields in terms of the ISI subject
> categories is very questionnable.
>
> However, the authors are very clear about their results: "We
> identified ...
> , but were unable to recognize a meaningful pattern in the
> results." (p.
> 9).
> I don't understand why one should then multiply the one
> measure with the other. What does multiplication to the error?
>
> Does one of you know a place where the ISI subject categories
> are justified?
> How are they produced? People seem to use them increasingly
> both in evaluation and research practices, but I have never
> been able to reproduce them using journal citation measures.
>
> With best wishes,
>
>
> Loet
>
>
> ________________________________
> Loet Leydesdorff
> Amsterdam School of Communications Research (ASCoR),
> Kloveniersburgwal 48, 1012 CX Amsterdam.
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
> loet at leydesdorff.net ; http://www.leydesdorff.net/
>
>
>
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Eugene Garfield
> > Sent: Friday, March 03, 2006 6:37 PM
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
> > Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1
> > 9 Jan 2006
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Further to yesterday's posting, "Prestige is factored into journal
> > ratings", here is another interesting and informative article
> >
> > FULL TEXT AVAILABLE AT :
> > http://www.arxiv.org/PS_cache/cs/pdf/0601/0601030.pdf
> >
> > email: {jbollen, marko, herbertv}@lanl.gov
> >
> > TITLE   :  Journal Status
> >
> > AUTHORS :  Johan Bollen, Marko A. Rodriguez, and Herbert
> Van de Sompel
> >
> > SOURCE   : arXiv:cs.GL/0601030 v1 9 Jan 2006
> >
> > Abstract
> > The status of an actor in a social context is commonly defined in
> > terms of two factors: the total number of endorsements the actor
> > receives from other actors and the prestige of the endorsing actors.
> > These two factors indicate the distinction between popularity and
> > expert appreciation of the actor, respectively. We refer to
> the former
>
> > as popularity and to the latter as prestige. These notions of
> > popularity and prestige also apply to the domain of scholarly
> > assessment. The ISI Impact Factor (ISI IF) is defined as the mean
> > number of citations a journal receives over a 2 year
> period. By merely
>
> > counting the amount of citations and disregarding the
> prestige of the
> > citing journals, the ISI IF is a metric of popularity, not of
> > prestige. We demonstrate how a weighted version of the popular
> > PageRank algorithm can be used to obtain a metric that reflects
> > prestige. We contrast the rankings of journals according to
> their ISI
> > IF and their weighted PageRank, and we provide an analysis that
> > reveals both significant overlaps and differences.
> > Furthermore, we introduce the Y-factor which is a simple
> combination
> > of both the ISI IF and the weighted PageRank, and find that the
> > resulting journal rankings correspond well to a general
> understanding
> > of journal status.
> >
> >
> > ______________________________________________
> >
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > FULL TEXT AVAILABLE AT :
> >
> http://www.nature.com/nature/journal/v439/n7078/pdf/439770a.pdf    OR
> > http://guide.labanimal.com/news/2006/060213/full/439770a.html
> >
> >
> > Philip Ball : p.ball at nature.com
> > www.philipball.com
> >
> > Title: Prestige is factored into journal ratings
> >
> > Author(s): Ball P
> >
> > Source: NATURE 439 (7078): 770-771 FEB 16 2006
> >
> > Document Type: News Item Language: English
> > Cited References: 0      Times Cited: 0
> >
> > Publisher: NATURE PUBLISHING GROUP, MACMILLAN BUILDING, 4
> CRINAN ST,
> > LONDON
> > N1 9XW, ENGLAND
> > Subject Category: MULTIDISCIPLINARY SCIENCES IDS Number: 012JA
> >
> > ISSN: 0028-0836
> >
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060306/dee55990/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fig12.jpg
Type: image/jpeg
Size: 36597 bytes
Desc: fig12.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060306/dee55990/attachment.jpg>