Person Citation Indexes unlimited!!!

[Garfield, Eugene]

  Bernie Sloan: I suppose that it is to be expected that after 35 years
personal citation reports should be rediscovered. I suppose that through
your eyes  Personal Citation Indexes represent a kind of epiphany. And I
will be grateful if your new look at personal awareness produces greater
attention to this potential of citation indexes and the web. And as Peter
Ingwerson suggests, it may cause even further and deeper understanding of
the meanings of one's impact.


When Bernie Sloan writes this all up I hope that he will be aware that there
have been thousands of individuals who have been compiling personal citation
reports for decades. Adding the new dimension of the web is an excellent
embellishment, but whether it has any significance for research, rather than
social interaction, is something to be seen. Many people other than
researchers have social impacts. As Blaise Cronin has been trying to
emphasize it would be nice to codify acknowledgements of all kinds and if
the web can call some of these to one's attention that is all to the good.
Teachers ought to be accorded the recognition that web links provide by
citing their courses. And if they can convert that into useful information
for tenure evaluation all the better.

I have written a great deal about personalized information services. Perhaps
the internet age will produce a more egocentric view of things and we can
get researchers to compile personal citation indexes. But long years of
experience demonstrates that it is difficult to get researchers to be that
systematic. Fewer of them today feel self-conscious about compiling such
reports. In the past it was wrongly interpreted as being vainglorious to
look up one's citation record. This unfortunately goes back to the early
days when citation indexes were still considered unwelcome by those who were
not cited very often--and this happens to be a significant part of the
academic population.

There isn't a week that goes by that I don't provide a personal citation
index to a researcher who has been active for decades. All that I have done
is to enter his or her name in the Web of Science to first identify their
published output and then to see how often and where they are cited. This
often produces an overwhelming feeling because after a lifetime of research
the information comes almost too late. It would have been much easier to
absorb the information in weekly chunks by using a personal citation alert,
but now they have to deal with dozens of papers never encountered before.
But then there is hope--for those who want to look back on the impact of
their work this personal report provides the basis for a lot of
interpretation and often the results are quite surprising.

It is the overwhelming nature of the information that has led me to pursue
in more recent times my interest in providing researchers with a tool that
takes this information and presents the personal citation report in a form
that is amenable to digestion.

I hope these ad hoc comments are what you are looking for. I hope you will
pursue your ideas on the personal citation index with vigor and inculcate in
students the importance of doing this on a regular and continuing basis.
With best  wishes. Gene Garfield



When responding, please attach my original message
__________________________________________________
Eugene Garfield, PhD. email:  garfield at codex.cis.upenn.edu
home page: www.eugenegarfield.org
Tel: 215-243-2205 Fax 215-387-1266
President, The Scientist LLC. www.the-scientist.com
Chairman Emeritus, ISI www.isinet.com
Past President, American Society for Information Science and Technology
(ASIS&T) www.asis.org






-----Original Message-----
From: Sloan, Bernie [mailto:bernies at UILLINOIS.EDU]
Sent: Tuesday, July 09, 2002 1:44 PM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] SV: [SIGMETRICS] Web citation issues


Peter,

For the past two years or so I have been dabbling in the collection of
citations and inlinks to several papers of mine. I use your definition of
"inlinks": "links pointing to web pages" (Bjorneborn & Ingwersen). I've been
using both publicly available Web search engines (Google and AllTheWeb), as
well as some commercial subscription-only databases from providers (e.g.,
ISI, the Gale Group, and EBSCO). I tried to be somewhat selective and did
some brief analysis of the items I retrieved, e.g., I did not count inlinks
from personal bookmark lists, or references to the papers from discussion
group archives.

I don't pretend that my papers are scientific papers, or that the citations
and inlinks I have discovered are for "scientific items citing other
scientific items." I did my best to eliminate any marginal entries, although
"marginal" is definitely in the eye of the beholder here. I like to think of
what I am doing as using the Web to discover the "influence of ideas". The
exercise started with the Web of Science, and then broadened out to include
the resources mentioned in the preceding paragraph. I don't think of the
exercise as something that replaces ISI databases such as SCI, SSI, etc.,
but rather something that supplements the information obtained from ISI.

One of the most interesting findings is that this exercise uncovered an
international influence that I would not have been aware of using only the
ISI databases. More than 40% of the citations/inlinks were from non-US
sources. Of the citations/inlinks from non-US sources, fully 60% were
associated with countries with a primary language other than English.

Right now I am working on putting together a paper reporting on my findings,
discoveries and questions in more detail, sort of a report on a
work-in-progress. The biggest question I am grappling with is "So what?" I
have a detailed listing of citations/inlinks for several of my papers.
Beyond satisfying personal and academic curiosity, what can these "personal
citation indices" be used for? Using them for evaluation springs to mind,
but of course "evaluation" requires comparison against some standard or
control group. I'm not aware of too many people who have conducted this
exercise at a personal level. I have something that seems impressive to me,
in a relative sense (i.e., a more comprehensive overview of the influence of
ideas than I might obtain from a citation index like SSI). But what can I
use it for?

Bernie Sloan

-----Original Message-----
From: Ingwersen Peter [mailto:PI at DB.DK]
Sent: Tuesday, July 09, 2002 9:52 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: [SIGMETRICS] SV: [SIGMETRICS] Web citation issues

Dear colleagues, as you may be well aware of our department was one of the
first to engage into webometric analyses, in particular of web impact by
means of inlinks. Several others around have done deep analyses of
webometric nature to see if, for instance, there exists a correlation
between citation impact and web impact/no. of inlinks (Bar Ilan) or between
other indicators of recognition (e.g. RAE in the UK) and web impact
(Thelwall). Sometimes there exist such correlations, but the major problem
is that the web acts differently than scientific communication vehicles (no
real conventions of linking etc.). Also: one may not equalize a web page
with a scientific article. So Tom Wilson´s no. of hits (to web pages) does
not necessarily correspond at all to "scientific items citing other
scientific items" as in conventional citation analyses. Only the reading
lists found by Wilson may act like such "reference lists". Further:
Different search engines commonly perform differently on the same search
profile, resulting in biased counts; several large quality studies have been
done on "scientific" web page search results to observe the actual
proportion of scientific output (see e.g. Allen et al in Science in 1999.)
Results are appalling and pauvre - and highly dependent on the domain in
question. For instance, in politically hot scientific topics (like in the
environmental sc.-) there is a chaotic mix of scientific, semi-scientific,
pseudo, popular and, foremost, political opinion papers. Should all the
"published" pages count or only the peer reviewed ones - e.g. those
published by scientific institutions or referring to peer reviewed journals
or published in peer reviewed e-journals?
Additionally, there are several possible web impacts: by inlink counts; by
inlink counts and outlink counts; by web-based traditional
references/citations on all the open web - or only in e-journals - with or
without peer review.
Tom will of course also run into the problem similar to that of ISI: in the
citation databases no citations from books and non-ISI journals are counted;
on the web only the open weblinks (and citations/references) are possible to
count - not links/citations provided by pages/items on the hidden web (e.g.
all the Dialog or ISI databases or the journals in publisher archives or in
Digital Libraries.
Finally, on the web the nature of obsolescence of information is quite
different and not yet well understood - see e.g. Ronald Rousseau´s articles
in Cybermetrics or recent publications by Wolfgang Glanzel on the issue.
A recent review on some of the issues touched upon above is: Bjorneborn, L.
& Ingwersen, P.: Perspectives of webometrics. Scientometrics, 50(1): 65-82.

Many regards - Peter Ingwersen
********************************************
Peter Ingwersen, Professor, Ph.D.
Department of Information Studies
Royal School of Library and Information Science,
Birketinget 6, DK 2300 Copenhagen S - Denmark
Tel: +45 32 58 60 66; FAX: +45 32 84 02 01
http://www.db.dk/pi/ - e-mail: pi at db.dk
Visiting Professor (Docent), Dept. of Information Studies
Åbo University Akademi - Finland
********************************************


-----Oprindelig meddelelse-----
Fra: Quentin L. Burrell [mailto:quentinburrell at MANX.NET]
Sendt: 21. juni 2002 18:11
Til: SIGMETRICS at LISTSERV.UTK.EDU
Emne: Re: [SIGMETRICS] Web citation (fwd)


I am wholly in support of the SIGMETRICS site being one for discussion and
so was interested in Tom's original submission and now Gene's response. Here
comes my two penn'orth (Eng., coll., obs.?)

Tom's observations are interesting - to his comment on token citations I
would add (many cases of) self-citation - but I go along with Gene's
uneasiness on the current haphazard coverage of the web being adequate to
replace formal citation bases.

Gene's final remark that the "ultimate objective of universal
bibliographical control is to find it all in one place, displayed in a
fashion that is easily and quickly comprehended" surely requires some
response.

(i)I guess that "control" was a hasty first attempt and that "information"
is more in line with the intended meaning.

(ii) I would really like to see the phrase "freely available" inserted
somewhere in the remark. At the moment, unless you are the member of a
subscribing institution you don't have free access to this bibliographic
information, either to "boost your ego" or to measure your impact.Citation
analysis - like any othe form of data analysis - requires access to the
data.

Anyone else willing to chip in a cent or a yen or a euro or a ... ?

Quentin Burrell


-----Original Message-----
From: ASIS Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU]On Behalf Of Garfield, Eugene
Sent: 19 June 2002 18:27
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Web citation (fwd)


It is almost six weeks since Tom Wilson posted this message. Many others
forwarded copies to me expecting me to respond to his challenge.

It is always pleasant to learn that one's work has been mentioned on a
particular web site or that it is discussed in various courses. But while
they are newsworthy they havoc little real bearing on the use of citation
indexes to measure the impact of one's research.

When you are quite young anything helps boost the ego, but the bottom line
for the researcher is whether anyone has used his or her basic ideas in
ongoing research. Until that day of Nirvana arrives when everything will be
searchable on the web I am afraid web searching just won't be an adequate
substitute.

If you are working in the life sciences you can find many relevant citation
connections through such full text resources as HighWire Press, but that is
not yet complete nor is it presented in a form that is easily used for
citation analysis. That day may come. Steve Lawrence's project at NEC which
provides citation indexing in context for the computer science literature
illustrate what happens when you have only partial coverage.

The ultimate objective of universal bibliographical control is to find it
all in one place, displayed in a fashion that is easily and quickly
comprehended. Gene Garfield

When responding, please attach my original message
__________________________________________________
Eugene Garfield, PhD. email:  garfield at codex.cis.upenn.edu
home page: www.eugenegarfield.org
Tel: 215-243-2205 Fax 215-387-1266
President, The Scientist LLC. www.the-scientist.com
Chairman Emeritus, ISI www.isinet.com
Past President, American Society for Information Science and Technology
(ASIS&T) www.asis.org






-----Original Message-----
From: Gretchen Whitney [mailto:gwhitney at UTK.EDU]
Sent: Monday, May 06, 2002 10:10 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: [SIGMETRICS] Web citation (fwd)


---------- Forwarded message ----------
Date: Mon, 06 May 2002 12:25:22 +0100
From: Prof. Tom Wilson <t.d.wilson at SHEF.AC.UK>
To: JESSE at LISTSERV.UTK.EDU
Subject: Web citation

There have been a few mentions of Web citation searching possibly replacing
citation indexing in time and I wondered how many people are now, as a
matter of course, using counts of Web mentions in their cases for
appointment, tenure or promotion.

I looked at a couple of my own papers and counted the SSCI citations and
then searched for mentions of the papers on the Web - the results left me
wondering whether the reliance on citation indexing as a measure of
performance is now past its sell by date.

My most cited paper is "On user studies and information needs" (1981) - a
Web search (using Google) revealed 118 pages that listed the title. The
pages were reading lists, free electronic journals, and documents that would
never be covered by SSCI, such as reports from various agencies. SSCI
revealed, if I recall aright, 79 citations of the paper. The question is: is
the Web revealing impact more effectively than SSCI?  Citation in scholarly
papers takes a variety of forms and much citation is of a token variety - x
is cited because x is always cited. On the other hand citation on reading
lists implies some positive recommendation of the text, and mention in
policy documents and the like, implies (at least in some cases) that some
benefit has been found in the cited document.

It may also be that the use of Web citation would provide a more complete
measure - I discovered, much to my surprise, that a 1971 text of mine on
'chain indexing' is cited on one reading list and in the bibliography of a
document in German on classification. Greater international coverage is a
further benefit of using Web citation.

It strikes me that a move towards using Web citation as the measure of
performance would be rather more useful than the use of citation indexes.

No doubt others have looked at this issue - is any consensus emerging?

Tom Wilson

-----------------------------------
Professor T.D. Wilson, PhD
Publisher/Editor in Chief
Information Research
University of Sheffield
Sheffield S10 2TN
United Kingdom
Tel: +44-114-222-2642
E-mail: t.d.wilson at shef.ac.uk
Web site: http://InformationR.net/
-----------------------------------