accuracy of Thomson data
Eugene Garfield
eugene.garfield at THOMSON.COM
Sat Dec 22 13:09:48 EST 2007
Dear Loet: The accuracy of "total cites" will improve when "journal
abbreviation spell checkers" are introduced in the publishing process.
This is technologically possible now with cheap terabyte memories. But
to advise publishers to avoid "twigging" journals or to avoid name
changes on the basis of this kind of "visibility" is nonsense. The
ultimate conclusion is to never start any new journals.
The "key-save" process described long ago already standardizes many
author journal abbreviation errors but was always limited by computer
memory considerations and the lack of complete legacy files.
"Journal Editors Awaken to the Impact of Citation Errors. How We Control
Them at ISI "
http://www.garfield.library.upenn.edu/essays/v13p367y1990.pdf
I believe it was Curtis Benjamin who invented the term twigging. Gene
-----Original Message-----
From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
Sent: Saturday, December 22, 2007 3:55 AM
To: SIGMETRICS at listserv.utk.edu
Subject: Re: [SIGMETRICS] accuracy of Thomson data
Dear Stephen,
Paradoxically, the 50,000+ yearly citations to "J Phys Chem-US" do not
affect so much the Impact Factor (because of its limitation to the last
two
years), but it does dramatically affect Total Cites. The journal was
split
into different parts in 1997. (Yesterday, the ACS launched J Phys Chem-C
with a focus on nano.)
The lesson from this seems to me a policy advice to scientific
publishers:
do not change the name of the journal! The journal (and potentially the
authors) will suffer in terms of visibility.
Best wishes,
Loet
________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
loet at leydesdorff.net ; http://www.leydesdorff.net/
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Stephen J Bensman
> Sent: Friday, December 21, 2007 6:10 PM
> To: SIGMETRICS at listserv.utk.edu
> Subject: Re: [SIGMETRICS] accuracy of Thomson data
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Loet,
> Thanks for the tip. I thought of that but that it requires
> knowing all
> the abbreviations used for the various parts. Then there was the
> problem of defining in what year the cites were made. They have to be
> limited to the JCR Year. At that point my mind boggled at the
> complexity of the process, and I thought perhaps it best to restrict
> oneself to what can be aggregated in the JCR and rely on the adequate
> sample theory that the JCRs first used. But I did not put a lot of
> time into this, because I am no longer interested in the
> Impact Factor.
>
> Thanks for sparing me the explanation of the "vector space
> models." The
> only vectors I know are compass ones used for triangulation
> purposes to
> knock out enemy artillery and other unhealthy things.
>
> Stephen J. Bensman
> LSU Libraries
> Louisiana State University
> Baton Rouge, LA 70803
> USA
> notsjb at lsu.edu
>
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
> Sent: Friday, December 21, 2007 10:31 AM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] accuracy of Thomson data
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Dear Stephen,
>
> As you know, I was interested in the so-called
> "externally-cited impact
> factor" of a journal which is not included in the set and then I
> realized
> that one can search on non-includied journals. In the
> Web-version, it is
> easy. You can just type "J PHYS CHEM-US" in the cited reference search
> or if
> you wish "WALL STREET J" and you get the citations.
>
> It seems virtually impossible for me for the ISI (or Scopus) to
> attribute
> the citations to "J Phys Chem-US" to either "J Phys Chem A" or "J Phys
> Chem
> B". The same holds true for the BBA-volumes, etc. I don't blame them,
> but
> one should be aware of the problem. This underestimates the citations,
> but
> as you know we found also 21.5% overestimation in the case of
> "Angewandte
> Chemie" because of people citing both editions. However, the group of
> journals for which this latter effect may happen, can be delimited.
>
> You don't mind if I am not going to explain the vector space model. I
> think
> the reference is Salton & McGill (1983) and if I remember
> correctly, the
> explanation is rather algebraic.
>
> With best wishes,
>
>
> Loet
>
> ________________________________
>
> Loet Leydesdorff
> Amsterdam School of Communications Research (ASCoR),
> Kloveniersburgwal 48, 1012 CX Amsterdam.
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
> loet at leydesdorff.net ; http://www.leydesdorff.net/
>
>
>
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J Bensman
> > Sent: Friday, December 21, 2007 4:01 PM
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: Re: [SIGMETRICS] accuracy of Thomson data
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Loet,
> > In re to your first paragraph, if you read the paper that
> is posted on
> > Dr. Garfield's Web site, you will see that the probability was very
> > stable across time both in terms of the Impact Factor and
> Total Cites
> > not at the aggregate level but within one
> discipline--chemistry. This
> > stability is essentially a reflection of the stability of the
> > underlying
> > social stratification system of chemistry. The only
> consistent change
> > in probability was the continuing increase in dominance of
> > the journals
> > at the top of both measures, indicating the success-breeds-success
> > mechanism of the Matthew Effect--technically known as "contagion" in
> > statistics. This is the basic reason for the stability of both the
> > social stratification system and the journal system based upon it.
> >
> > In re your method, I must admit that generally it shoots
> right by me.
> > When I was young, a mathematics teacher once told me that
> minds can be
> > divided into two types--those that think linearly and are good in
> > algebra, and those that think spatially and are good in
> geometry. The
> > teacher said that people very good in algebra often have
> hard time in
> > geometry. I was extremely good in algebra but had a hard time in
> > geometry due to difficulty in grasping spatial relationships.
> > It seems
> > that computer programmers have to be able to think
> spatially as well,
> > and I have a hard time understanding this.
> >
> > However, your most important recent discovery--in my opinion--was in
> > your recent "Caveats" paper, where you prove that JCRs do not
> > aggregate
> > Total Cites across title changes, splits, etc. In other
> > words, much of
> > the backfile is left out of the count. From my historical
> > perspective,
> > this is a most serious flaw. You also have to understand
> that I am a
> > catalog librarian and define serial bibliographic entities
> in terms of
> > volume count--if the volume sequence is consistent, it is
> > still the same
> > journal despite the title change. I investigated the
> problem briefly
> > and found that I could not aggregate more than three years. I would
> > like to know how you did that. For me it means that for
> > Total Cites to
> > be good, Total Cites across 3 years have to be highly
> correlated with
> > Total Cites across the entire backfile. An initial view was
> > that such a
> > short backfile was a sufficient sample. If this is so,
> then the Total
> > Cites measure is good.
> >
> > Stephen J. Bensman
> > LSU Libraries
> > Louisiana State University
> > Baton Rouge, LA 70803
> > USA
> > notsjb at lsu.edu
> >
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
> > Sent: Friday, December 21, 2007 1:05 AM
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: Re: [SIGMETRICS] accuracy of Thomson data
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Dear Stephen and colleagues,
> >
> > These (Spearman) correlations teach us, in my opinion, that
> > the measures
> > covary across disciplines at the aggregate level. Since
> impact factors
> > are
> > higher in the biomedical sciences, library usages and expert ratings
> > would
> > be expected to be higher in these sciences as well?
> >
> > I would expect the c/p ratio, the impact factor, and the immediacy
> > factor to
> > correlate highly as one group, but total publication, total
> citations,
> > library usage, expert ratings, etc., as a second group (size
> > related).
> >
> > As you know, my preference goes in the direction of mapping local
> > citation
> > impact environments. These can be visualized using, for
> example, the
> > files
> > at http://www.leydesdorff.net/jcr06 . (See: Visualization of the
> > Citation
> > Impact Environments of Scientific Journals: An online mapping
> > exercise,
> > Journal of the American Society for Information Science and
> Technology
> > 58(1), 25-38, 2007.) The normalization in terms of the vector space
> > (cosine)
> > takes care of the size effects in the relations (links), and
> > size can be
> > considered as an attribute of the nodes.
> >
> > I have no better operationalization at the moment, but I am open for
> > suggestions.
> >
> > Best wishes,
> >
> >
> > Loet
> >
> > ________________________________
> >
> > Loet Leydesdorff
> > Amsterdam School of Communications Research (ASCoR),
> > Kloveniersburgwal 48, 1012 CX Amsterdam.
> > Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
> > loet at leydesdorff.net ; http://www.leydesdorff.net/
> >
> >
> >
> > > -----Original Message-----
> > > From: ASIS&T Special Interest Group on Metrics
> > > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen
> J. Bensman
> > > Sent: Thursday, December 20, 2007 10:37 PM
> > > To: SIGMETRICS at LISTSERV.UTK.EDU
> > > Subject: Re: [SIGMETRICS] accuracy of Thomson data
> > >
> > > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > > http://web.utk.edu/~gwhitney/sigmetrics.html
> > >
> > > I try to do some of this in the paper posted on Dr.
> > > Garfield's Web site at:
> > >
> > >
> > >
> > > http://garfield.library.upenn.edu/bensman/bensmanegif22007.pdf
> > >
> > >
> > >
> > > You might want to look at the second half of the paper, where
> > > I discuss
> > > the Impact Factor in terms of Poisson lambdas, sampling
> > > variance, random
> > > error, etc. The amazing thing to me, at least, is that
> > > despite all the
> > > random error and sampling variance, there is a remarkable
> > > stability of
> > > probability across time with Spearman rhos of 0.9 and above
> > with high
> > > respectable correlations with Total Cites, library use,
> and expert
> > > ratings. Most impact factors move up and down within
> > > extremely narrow
> > > limits across time. I found a similar phenomenon in a paper
> > > just accepted
> > > by JASIST called "Distributional Differences of the Impact
> > > Factor in the
> > > Sciences vs. the Social Sciences: An Analysis of the
> Probabilistic
> > > Structure of the 2005 Journal Citation Reports." I no
> > > longer own the
> > > copyright and so cannot post it, but I suppose that I can let
> > > you read it
> > > on a private basis, if you're willing to suffer the pain of
> > > reading it.
> > > There is much more to the Impact Factor than meets the eye,
> > > and it is an
> > > extremely good measure for many purposes, if of extremely
> > > doubtful use for
> > > ranking purposes in the vast bulk of the cases.
> > >
> > >
> > >
> > > Stephen J. Bensman, Ph.D.
> > > LSU Libraries
> > > Louisiana State University
> > > Baton Rouge, LA 70803
> > > USA
> > > notsjb at lsu.edu
> > >
> >
>
More information about the SIGMETRICS
mailing list