SV: [SIGMETRICS] Web citation issues

[Ingwersen Peter]

Dear colleagues, as you may be well aware of our department was one of the
first to engage into webometric analyses, in particular of web impact by
means of inlinks. Several others around have done deep analyses of
webometric nature to see if, for instance, there exists a correlation
between citation impact and web impact/no. of inlinks (Bar Ilan) or between
other indicators of recognition (e.g. RAE in the UK) and web impact
(Thelwall). Sometimes there exist such correlations, but the major problem
is that the web acts differently than scientific communication vehicles (no
real conventions of linking etc.). Also: one may not equalize a web page
with a scientific article. So Tom Wilson´s no. of hits (to web pages) does
not necessarily correspond at all to "scientific items citing other
scientific items" as in conventional citation analyses. Only the reading
lists found by Wilson may act like such "reference lists". Further:
Different search engines commonly perform differently on the same search
profile, resulting in biased counts; several large quality studies have been
done on "scientific" web page search results to observe the actual
proportion of scientific output (see e.g. Allen et al in Science in 1999.)
Results are appalling and pauvre - and highly dependent on the domain in
question. For instance, in politically hot scientific topics (like in the
environmental sc.-) there is a chaotic mix of scientific, semi-scientific,
pseudo, popular and, foremost, political opinion papers. Should all the
"published" pages count or only the peer reviewed ones - e.g. those
published by scientific institutions or referring to peer reviewed journals
or published in peer reviewed e-journals?
Additionally, there are several possible web impacts: by inlink counts; by
inlink counts and outlink counts; by web-based traditional
references/citations on all the open web - or only in e-journals - with or
without peer review.
Tom will of course also run into the problem similar to that of ISI: in the
citation databases no citations from books and non-ISI journals are counted;
on the web only the open weblinks (and citations/references) are possible to
count - not links/citations provided by pages/items on the hidden web (e.g.
all the Dialog or ISI databases or the journals in publisher archives or in
Digital Libraries.
Finally, on the web the nature of obsolescence of information is quite
different and not yet well understood - see e.g. Ronald Rousseau´s articles
in Cybermetrics or recent publications by Wolfgang Glanzel on the issue.
A recent review on some of the issues touched upon above is: Bjorneborn, L.
& Ingwersen, P.: Perspectives of webometrics. Scientometrics, 50(1): 65-82.

Many regards - Peter Ingwersen
********************************************
Peter Ingwersen, Professor, Ph.D.
Department of Information Studies
Royal School of Library and Information Science,
Birketinget 6, DK 2300 Copenhagen S - Denmark
Tel: +45 32 58 60 66; FAX: +45 32 84 02 01
http://www.db.dk/pi/ - e-mail: pi at db.dk
Visiting Professor (Docent), Dept. of Information Studies
Åbo University Akademi - Finland
********************************************


-----Oprindelig meddelelse-----
Fra: Quentin L. Burrell [mailto:quentinburrell at MANX.NET]
Sendt: 21. juni 2002 18:11
Til: SIGMETRICS at LISTSERV.UTK.EDU
Emne: Re: [SIGMETRICS] Web citation (fwd)


I am wholly in support of the SIGMETRICS site being one for discussion and
so was interested in Tom's original submission and now Gene's response. Here
comes my two penn'orth (Eng., coll., obs.?)

Tom's observations are interesting - to his comment on token citations I
would add (many cases of) self-citation - but I go along with Gene's
uneasiness on the current haphazard coverage of the web being adequate to
replace formal citation bases.

Gene's final remark that the "ultimate objective of universal
bibliographical control is to find it all in one place, displayed in a
fashion that is easily and quickly comprehended" surely requires some
response.

(i)I guess that "control" was a hasty first attempt and that "information"
is more in line with the intended meaning.

(ii) I would really like to see the phrase "freely available" inserted
somewhere in the remark. At the moment, unless you are the member of a
subscribing institution you don't have free access to this bibliographic
information, either to "boost your ego" or to measure your impact.Citation
analysis - like any othe form of data analysis - requires access to the
data.

Anyone else willing to chip in a cent or a yen or a euro or a ... ?

Quentin Burrell


-----Original Message-----
From: ASIS Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU]On Behalf Of Garfield, Eugene
Sent: 19 June 2002 18:27
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Web citation (fwd)


It is almost six weeks since Tom Wilson posted this message. Many others
forwarded copies to me expecting me to respond to his challenge.

It is always pleasant to learn that one's work has been mentioned on a
particular web site or that it is discussed in various courses. But while
they are newsworthy they havoc little real bearing on the use of citation
indexes to measure the impact of one's research.

When you are quite young anything helps boost the ego, but the bottom line
for the researcher is whether anyone has used his or her basic ideas in
ongoing research. Until that day of Nirvana arrives when everything will be
searchable on the web I am afraid web searching just won't be an adequate
substitute.

If you are working in the life sciences you can find many relevant citation
connections through such full text resources as HighWire Press, but that is
not yet complete nor is it presented in a form that is easily used for
citation analysis. That day may come. Steve Lawrence's project at NEC which
provides citation indexing in context for the computer science literature
illustrate what happens when you have only partial coverage.

The ultimate objective of universal bibliographical control is to find it
all in one place, displayed in a fashion that is easily and quickly
comprehended. Gene Garfield

When responding, please attach my original message
__________________________________________________
Eugene Garfield, PhD. email:  garfield at codex.cis.upenn.edu
home page: www.eugenegarfield.org
Tel: 215-243-2205 Fax 215-387-1266
President, The Scientist LLC. www.the-scientist.com
Chairman Emeritus, ISI www.isinet.com
Past President, American Society for Information Science and Technology
(ASIS&T) www.asis.org






-----Original Message-----
From: Gretchen Whitney [mailto:gwhitney at UTK.EDU]
Sent: Monday, May 06, 2002 10:10 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: [SIGMETRICS] Web citation (fwd)


---------- Forwarded message ----------
Date: Mon, 06 May 2002 12:25:22 +0100
From: Prof. Tom Wilson <t.d.wilson at SHEF.AC.UK>
To: JESSE at LISTSERV.UTK.EDU
Subject: Web citation

There have been a few mentions of Web citation searching possibly replacing
citation indexing in time and I wondered how many people are now, as a
matter of course, using counts of Web mentions in their cases for
appointment, tenure or promotion.

I looked at a couple of my own papers and counted the SSCI citations and
then searched for mentions of the papers on the Web - the results left me
wondering whether the reliance on citation indexing as a measure of
performance is now past its sell by date.

My most cited paper is "On user studies and information needs" (1981) - a
Web search (using Google) revealed 118 pages that listed the title. The
pages were reading lists, free electronic journals, and documents that would
never be covered by SSCI, such as reports from various agencies. SSCI
revealed, if I recall aright, 79 citations of the paper. The question is: is
the Web revealing impact more effectively than SSCI?  Citation in scholarly
papers takes a variety of forms and much citation is of a token variety - x
is cited because x is always cited. On the other hand citation on reading
lists implies some positive recommendation of the text, and mention in
policy documents and the like, implies (at least in some cases) that some
benefit has been found in the cited document.

It may also be that the use of Web citation would provide a more complete
measure - I discovered, much to my surprise, that a 1971 text of mine on
'chain indexing' is cited on one reading list and in the bibliography of a
document in German on classification. Greater international coverage is a
further benefit of using Web citation.

It strikes me that a move towards using Web citation as the measure of
performance would be rather more useful than the use of citation indexes.

No doubt others have looked at this issue - is any consensus emerging?

Tom Wilson

-----------------------------------
Professor T.D. Wilson, PhD
Publisher/Editor in Chief
Information Research
University of Sheffield
Sheffield S10 2TN
United Kingdom
Tel: +44-114-222-2642
E-mail: t.d.wilson at shef.ac.uk
Web site: http://InformationR.net/
-----------------------------------