Query about journal (not author) self-citation rates

Wed Mar 26 02:23:23 EST 2003

Dear Stevan,

1. There are standardized ways to normalize outlayers on the main
diagonal because that problem is occuring more frequently.
Self-citations can be considered as the values on the main diagonal of
the citation matrix.

Note that the distributions are not normal and that one should therefore
preferentially use non-parametric statistics for the normalization.

* Derek de Solla Price, "The Analysis of Square Matrices of
Scientometric Transactions," Scientometrics 3 (1981), 55-63.

* Elliott Noma, "An Improved Method for Analyzing Square Scientometric
Transaction Matrices," Scientometrics 4 (1982) 297-316.

2. Even if properly normalized the interpretation remains problematic.
Citations are not only higher within specific journals, but also within
specific groups of journals. This reflects restricted discourses (versus
elaborate discourses that reach across disciplinary delineations, e.g.,
Science and Nature). Within each group one can construct a ranking, but
the comparisons among groups in terms of citation rankings is
problematic. For example, impact factors are higher in immunology than
in toxicology. Furthermore, the delineations of the groupings is
uncertain and dynamic.

3. Citations are a mixed bag (see my article with Olga Amsterdamska,
"Dimensions of Citation Analysis," STHV 15 (1990) 305-335 and later work
on the theory of citations). One may wish to argue that at the level of
journals the law of large numbers does the job. But which job? Imho, the
job of grouping, but not of ranking.

This is not meant to discourage you from your project which is
interesting. The data can easily be collected from the CD-Rom version of
the JCR.

With kind regards,

Loet

  _____

Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
loet at leydesdorff.net ; http://www.leydesdorff.net/

The Challenge of Scientometrics ; The Self-Organization of the
Knowledge-Based Society

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stevan Harnad
> Sent: Wednesday, March 26, 2003 1:42 AM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] Query about journal (not author)
> self-citation rates
>
>
> On Tue, 25 Mar 2003, Bob Parks wrote:
>
> > You (Stevan) ought to be able to get to the JCR product on
> > http://wos.mimas.ac.uk/jcrweb/
>
> The UK national site license allows human access but not
> software-agent access, which is what we need (not just for
> this, but for a variety of other impact-related studies we
> want to do). We have been discussing the possibility of a
> collaborative research project with ISI, however, and if this
> goes forward then we could have a look at this too.
>
> > In www.iaes.org/journal/aej/dec_01/liner_pdf.pdf,
> > they correct for self-citations
>
> I believe that's only for author self-citations, not for
> same-journal citations.
>
> >sh>As a measure of "degree of egocentricity" relative to overall
> >sh>in/out/self citation patterns I (who am not a
> statistician!) would
> >sh>at first be inclined to look at the mean and standard
> error for the
> >sh>ratio:  S = self/(in + out)
> >sh>                T = in/(in + out)
> >sh>and then for:   1/(T-S) as a rough measure of egocentricity.
> >
> > Well that is (in +out)/(in - self) and I am not sure what
> that really
> > means.
> >
> > I would think that self/(in + out + self)  would be a better first
> > try.  Near 1 is very egocentric and near 0 is not very
> egocentric...
> > [but] I don't think ONE summary measure can capture it very well,
> > given the three in/out/self types.
>
> I agree that one measure will not be sensitive enough. We
> will experiment with this. (I think that "self" is part of
> "in" by the way, though I may be wrong.)
>
> >sh>It goes without saying that once the journal literature is
> >sh>open-access, potential journal-based biases like this will be far
> >sh>less consequential
> >
> > HUH?  Why?  If we have OA (complete, universal, all refereed
> > articles), AND we have journals, then why would journals (in that
> > utopian future) change their current biases?
>
> I said it would be far less consequential. It is
> consequential in the toll-access era, because journal impact
> factors partly determine which journals are subscribed to
> (licensed) by institutions, and therefore they partly
> determine what we do and do not have (toll)-access to. When
> all the annual 2,000,000 papers in all 20,000 refereed
> journals are self-archived and openly accessible to all
> potential users web-wide, whether or not their institutions
> can afford a subscription (license) to the toll-access
> version, then it *matters* far less what the journal impact
> factor happens to be, whether or not it has been inflated,
> and whether or not a given institution, as a result, subscribes to
> (licenses) the toll-access version. The self-archived
> open-access version of everything is available to all
> would-be users in any case.
>
> Moreover, the scientometric correction for any inflated
> same-journal citations could even be corrected in authors'
> *personal* citation counts, if desired. The open database
> could be used in a much more powerful and flexible way by all
> users and evaluators of research productivity. (In other
> words, I may lose a few citation because they are detectably
> just part of the inflated same-journal citations of some
> journal they have appeared in. (But I hope you agree that
> this bit of fine-tuning is not likely to be very
> consequential either.)
>
> > So, is the 'consequential' bias "there is every temptation to get
> > those journal impact factors as high as possible" and that would go
> > away in OA? If journals serve the same purpose in the UOA
> (Utopian OA)
> > as they do now, won't that temptation be the same?
>
> The core purpose served by journals in the UOA (universal
> open access) era will be exactly the same as it is now: to
> provide peer-review and to certify publication standards as
> having been successfully met (at that journal's established
> quality level). This is the core "publish-or-perish"
> function, and it remains unchanged. How long the further
> journal publication functions (paper version, publisher's
> online PDF, dissemination, storage, access-provision) will
> continue to be needed (and hence paid for) is not something
> that I or anyone can or need guess.
> http://www.nature.com/nature/debates/e-access/Articles/harnad.html#B1
>
> All that needs to be understood is that once there is open
> access, no potential usage or impact will be lost because of
> inability to pay access tolls: and that is the *only* think
> open-access is about.
> http://www.ecs.soton.ac.uk/~harnad/Temp/unto-> others.html
>
>
> (And, as I said, if there is still any residual
> temptation to inflate journal impact through same-journal
> citation, it will matter a good deal less, and will be a lot
> more detectable and correctable.)
>
> >sh>because there will be many direct measures of a paper's
> or author's
> >sh>research impact, among which the citation impact factor of the
> >sh>journal in which the paper appeared will be a relatively
> minor one.
> >
> > Well, I can only think within my own profession.  If journals are
> > around in the UOA, I think that their 'rankings' will be about the
> > same and for the same reasons.  Some will get higher, some
> lower, but
> > for the most part they will remain the same.
>
> But who will care, since it is the impact of the research and
> the researcher that matters, not the impact of the journal
> (which is merely the average impact of the papers it publishes)?
>
> > I don't see why what journal the article appeared in as
> being a minor
> > measure.  The current situation is based on the referee system and
> > self selection.  Top journals have top referees and get top
> articles.
> > UOA will not lessen that, and I doubt that dept chairs, or
> deans will
> > think that an article in a third tier journal is worth much even if
> > all of the other 'direct measures' available in UOA are high.
>
> But I agree completely! The top journals (i.e., the ones
> exercising the most rigorous peer review and selectivity,
> hence maintaining the highest quality standards) will
> continue to be given due weight for that -- along with the
> weight coming from the article's and the author's various
> measures of research impact (usage ["hits"], citations,
> "authority co-citations," etc.). Research impact will not be
> estimated by just the one-dimensional measure consisting of
> the journal's average citation count, but by a rich and
> diverse regression equation, with multiple weighted
> predictors. http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving.htm
>
> Stevan Harnad
>