Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Tue Mar 7 02:20:29 EST 2006

Dear Kevin,

My only point was that I would like to know how these categories are
assigned. I agree that some of these assigments are better than others and
that it is a difficult job, but we cannot control anything until we know the
how. Is there an algorithm, specific expertise or is it just done by one or
more indexers?

If you look for a better classification, one could use the one of the
Library of Congress. That is done very professionally.

With best wishes,

Loet

________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
loet at leydesdorff.net ; http://www.leydesdorff.net/

  _____

From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Boyack, Kevin W
Sent: Monday, March 06, 2006 11:01 PM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert Van
de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Dear Loet,

Your choice of category to highlight provides for some very interesting
points. "Biochemistry and Molecular Biology" is a very large and diverse
grouping in the ISI data. Journals are jointly categorized in this category
with some 59 other categories, which fact by itself says that this category
is too broad - thus supporting your argument. The largest multiple
categorizations are (2002 data) with:

CELL BIOLOGY    71
BIOPHYSICS    45
BIOTECH & APPLIED MICRO    24
GENETICS & HEREDITY    23
PHARMACOLOGY & PHARMACY    23
BIOCHEM RESEARCH METHODS    16
ENDOCRINOLOGY & METABOLISM    15
PLANT SCIENCES    13
IMMUNOLOGY    11
and so on.

But this also says to me that your comparison below is showing a bit of
"apples and oranges". For instance, one would not expect journals that are
jointly categorized BIOCHEM / BIOPHYSICS or PHARMA or PLANT SCIENCES ... to
appear in the overlap set of your venn diagrams. Your factor analysis
categorizations would have put these journals in factors different than
those you highlight below (JACS-related and JBC-related), and it is not
entirely fair to include them. If you were to limit your ISI comparison set
to journals either singly categorized as BIOCHEM or to those plus journals
doubly categorized to BIOCHEM + CELL BIOLOGY, your pictures would look much
different. I would love to see that analysis. When I look at the
distribution of the ISI categories in my cluster maps, and see that, for the
most part, 80% of the members of each cluster belong to a dominant ISI
category, I continue to retain my optimism. Regarding how categories are
assigned - I am perhaps too much of an optimist, but I have the hope that
whoever is doing it is smart, and is relying on some data to make judgments.

You are right about the pitfalls of using this categorization blindly - I
share your concerns. But people will increasingly use it until something is
shown to be better, and becomes readily available and publicly accepted.
Your local citation environments are a great step forward. I'd love to see
them in more common use.

Best regards,
Kevin

  _____

From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
Sent: Monday, March 06, 2006 12:54 PM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert Van
de Sompel "Journal Status" arXiv:cs.GL/0601030 v1 9 Jan 2006

Dear Kevin,

The above figure is from my paper entitled
<http://www.leydesdorff.net/classif03/index.htm> Can Scientific Journals be
Classified in terms of Aggregated Journal-Journal Citation Relations using
the Journal Citation Reports? Journal of the American Society for
Information Science and Technology (forthcoming). <
<http://www.leydesdorff.net/classif03/classif03.pdf> pdf-version> . It made
me less optimistic than you about the quality of the classification and I
asked therefore one colleague at ISI how this classification was made, and
he answered that he thought that there was no serious analytical basis for
it. My question at this list was more to ask whether someone knew the basis
of it than for being reassured.

It seems very worrysome to me that this classification is increasingly used
while it seems based on ....? I cannot find any confirmation. Your inference
that the discussion about inclusion warrants the quality of the
classification is also not warranted by research.

With best wishes,

Loet

________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
loet at leydesdorff.net ; http://www.leydesdorff.net/

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Boyack, Kevin W
> Sent: Monday, March 06, 2006 4:42 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez,
> and Herbert Van de Sompel "Journal Status"
> arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Hello everyone,
>
> Regarding ISI subject categories, I don't know how they are
> picked, but you can read about journal inclusion at
> http://scientific.thomson.com/free/essays/selectionofmaterial/
> journalsel
> ection/. I would assume that if this much effort is going
> into picking journals to index, there is at least some level
> of thought going into subject categorization. After all, the
> things ISI is looking at to decide on journal inclusion are
> in many cases the same things you would look at for categorization.
>
> Now I won't claim that ISI's categorization is perfect, or
> that it is updated as frequently as it might be, but I have
> yet to see anyone put forth a better categorization scheme
> that is widely available. I'm sure there are many of us that
> either have, or could easily, make up our own schemes based
> on citation patterns or other factors, that we feel would be
> more accurate.
>
> Now I'll go out on a limb with an opinion that is counter to
> what I see in many papers. On the whole I think the ISI
> categories are pretty good.
> There are some categories that should be thrown out, some
> that should be split, some that could be merged, and some
> journals that are clearly mis-categorized, as mentioned in my
> recent Scientometrics [v64(3), p351-374] paper with Klavans
> and Borner. We can have lots of discussion about the details,
> but when you look at the whole thing, it's a good start.
>
> Perhaps I'm more comfortable than most with the ISI structure
> because I don't rely on it for evaluation. We use the
> emergent cluster structure from papers rather than any
> category or journal scheme. But that's another story.
>
> Can anyone recommend a better journal categorization scheme
> that is widely available? I'd love to hear of (and get a copy
> of) such a thing.
>
> Best regards,
> Kevin
>
>
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
> Sent: Saturday, March 04, 2006 6:15 AM
> To: SIGMETRICS at listserv.utk.edu
> Subject: Re: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez,
> and Herbert Van de Sompel "Journal Status"
> arXiv:cs.GL/0601030 v1 9 Jan 2006
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Dear colleagues,
>
> The idea is interesting. However, there a few problems with
> this paper.
> First, the authors should not have used Pearson correlation
> coefficients to compare the rankings, but rank correlations
> (Spearman's rho or Kendall's tau). Second, it would have been
> interesting to have a rank correlation with "total cites"
> given recent discussions (Bensman).
> Third, the delineation of fields in terms of the ISI subject
> categories is very questionnable.
>
> However, the authors are very clear about their results: "We
> identified ...
> , but were unable to recognize a meaningful pattern in the
> results." (p.
> 9).
> I don't understand why one should then multiply the one
> measure with the other. What does multiplication to the error?
>
> Does one of you know a place where the ISI subject categories
> are justified?
> How are they produced? People seem to use them increasingly
> both in evaluation and research practices, but I have never
> been able to reproduce them using journal citation measures.
>
> With best wishes,
>
>
> Loet
>
>
> ________________________________
> Loet Leydesdorff
> Amsterdam School of Communications Research (ASCoR),
> Kloveniersburgwal 48, 1012 CX Amsterdam.
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
> loet at leydesdorff.net ; http://www.leydesdorff.net/
>
>
>
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Eugene Garfield
> > Sent: Friday, March 03, 2006 6:37 PM
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: [SIGMETRICS] Johan Bollen, Marko A. Rodriguez, and Herbert
> > Van de Sompel "Journal Status" arXiv:cs.GL/0601030 v1
> > 9 Jan 2006
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Further to yesterday's posting, "Prestige is factored into journal
> > ratings", here is another interesting and informative article
> >
> > FULL TEXT AVAILABLE AT :
> > http://www.arxiv.org/PS_cache/cs/pdf/0601/0601030.pdf
> >
> > email: {jbollen, marko, herbertv}@lanl.gov
> >
> > TITLE   :  Journal Status
> >
> > AUTHORS :  Johan Bollen, Marko A. Rodriguez, and Herbert
> Van de Sompel
> >
> > SOURCE   : arXiv:cs.GL/0601030 v1 9 Jan 2006
> >
> > Abstract
> > The status of an actor in a social context is commonly defined in
> > terms of two factors: the total number of endorsements the actor
> > receives from other actors and the prestige of the endorsing actors.
> > These two factors indicate the distinction between popularity and
> > expert appreciation of the actor, respectively. We refer to
> the former
>
> > as popularity and to the latter as prestige. These notions of
> > popularity and prestige also apply to the domain of scholarly
> > assessment. The ISI Impact Factor (ISI IF) is defined as the mean
> > number of citations a journal receives over a 2 year
> period. By merely
>
> > counting the amount of citations and disregarding the
> prestige of the
> > citing journals, the ISI IF is a metric of popularity, not of
> > prestige. We demonstrate how a weighted version of the popular
> > PageRank algorithm can be used to obtain a metric that reflects
> > prestige. We contrast the rankings of journals according to
> their ISI
> > IF and their weighted PageRank, and we provide an analysis that
> > reveals both significant overlaps and differences.
> > Furthermore, we introduce the Y-factor which is a simple
> combination
> > of both the ISI IF and the weighted PageRank, and find that the
> > resulting journal rankings correspond well to a general
> understanding
> > of journal status.
> >
> >
> > ______________________________________________
> >
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > FULL TEXT AVAILABLE AT :
> >
> http://www.nature.com/nature/journal/v439/n7078/pdf/439770a.pdf    OR
> > http://guide.labanimal.com/news/2006/060213/full/439770a.html
> >
> >
> > Philip Ball : p.ball at nature.com
> > www.philipball.com
> >
> > Title: Prestige is factored into journal ratings
> >
> > Author(s): Ball P
> >
> > Source: NATURE 439 (7078): 770-771 FEB 16 2006
> >
> > Document Type: News Item Language: English
> > Cited References: 0      Times Cited: 0
> >
> > Publisher: NATURE PUBLISHING GROUP, MACMILLAN BUILDING, 4
> CRINAN ST,
> > LONDON
> > N1 9XW, ENGLAND
> > Subject Category: MULTIDISCIPLINARY SCIENCES IDS Number: 012JA
> >
> > ISSN: 0028-0836
> >
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060307/139e04e0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fig12.jpg
Type: image/jpeg
Size: 36597 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060307/139e04e0/attachment.jpg>