Web citation issues

Wed Jul 10 08:51:25 EDT 2002

Hi Bernie and other interested parties,

I quite like the approach Bernie has taken. I appreciate that some of our
suggestions on link structures are useful to people. But most important,
Bernie´s approach emphasizes that the traditional (scientific) citation
analyses of incoming citations (not references) are of different nature
compared to web-citation/inlink analyses, since the latter probably
demonstrate a much wider dimensionality of "use" or recognition by others of
some work or manifestation published on the web. His conceptions of
"influence of ideas" is highly appropriate in that sense.

Since you Bernie has started up with some web publications well-known to you
(your own!) the observed inlinks pointing to them ought to be categorized if
possible by you Bernie and the results af that categorisation published.
That might be a starting point for providing better hypotheses about why
people/things on the web obtain inlinks, i.e forming a sort of inlink
theory. This is NOT the same as an outlink theory about why people generate
links. This phenomenon is similar to the difference between a reference
theory on why scientists make references (also often called a "citation
theory" unfortunately, and quite difficult to achive, according to Cronin &
others) and a proper citation theory, i.e., a theory on why people or
objects receive citations. It is commonly the latter (received citations or
received inlinks) that are counted in our analyses. As stated by van Raan in
the citation theory discussions in Scientometrics some years back, citation
analyses are about measuring impact of some sort, regardless why the
references were given in the first place.
So Bernie (and others) just keep going. Aside from the probably categories
you may find in your personal case, I predict that other categories may also
appear, due to differences in subject matter,
scientific/professional/cultural field, type of web site/object, etc. under
analysis for inlinks.
A future goal might then be to operate with several "impact indicators" on
the web: from strict scientific (citation) impact (similar to that from ISI)
over citation/inlink impact on/from scientific web-based material to all
kinds of inlinks, provided that such "kinds" are known and understood. Thats
your contribution. So long for a while due to holidays and conference
participation - yours Peter Ingwersen

********************************************
Peter Ingwersen, Professor, Ph.D.
Department of Information Studies
Royal School of Library and Information Science,
Birketinget 6, DK 2300 Copenhagen S - Denmark
Tel: +45 32 58 60 66; FAX: +45 32 84 02 01
http://www.db.dk/pi/ - e-mail: pi at db.dk
Visiting Professor (Docent), Dept. of Information Studies
Åbo University Akademi - Finland
********************************************

-----Oprindelig meddelelse-----
Fra: Sloan, Bernie [mailto:bernies at UILLINOIS.EDU]
Sendt: 9. juli 2002 19:44
Til: SIGMETRICS at LISTSERV.UTK.EDU
Emne: Re: [SIGMETRICS] SV: [SIGMETRICS] Web citation issues

Peter,

For the past two years or so I have been dabbling in the collection of
citations and inlinks to several papers of mine. I use your definition of
"inlinks": "links pointing to web pages" (Bjorneborn & Ingwersen). I've been
using both publicly available Web search engines (Google and AllTheWeb), as
well as some commercial subscription-only databases from providers (e.g.,
ISI, the Gale Group, and EBSCO). I tried to be somewhat selective and did
some brief analysis of the items I retrieved, e.g., I did not count inlinks
from personal bookmark lists, or references to the papers from discussion
group archives.

I don't pretend that my papers are scientific papers, or that the citations
and inlinks I have discovered are for "scientific items citing other
scientific items." I did my best to eliminate any marginal entries, although
"marginal" is definitely in the eye of the beholder here. I like to think of
what I am doing as using the Web to discover the "influence of ideas". The
exercise started with the Web of Science, and then broadened out to include
the resources mentioned in the preceding paragraph. I don't think of the
exercise as something that replaces ISI databases such as SCI, SSI, etc.,
but rather something that supplements the information obtained from ISI.

One of the most interesting findings is that this exercise uncovered an
international influence that I would not have been aware of using only the
ISI databases. More than 40% of the citations/inlinks were from non-US
sources. Of the citations/inlinks from non-US sources, fully 60% were
associated with countries with a primary language other than English.

Right now I am working on putting together a paper reporting on my findings,
discoveries and questions in more detail, sort of a report on a
work-in-progress. The biggest question I am grappling with is "So what?" I
have a detailed listing of citations/inlinks for several of my papers.
Beyond satisfying personal and academic curiosity, what can these "personal
citation indices" be used for? Using them for evaluation springs to mind,
but of course "evaluation" requires comparison against some standard or
control group. I'm not aware of too many people who have conducted this
exercise at a personal level. I have something that seems impressive to me,
in a relative sense (i.e., a more comprehensive overview of the influence of
ideas than I might obtain from a citation index like SSI). But what can I
use it for?

Bernie Sloan

-----Original Message-----
From: Ingwersen Peter [mailto:PI at DB.DK]
Sent: Tuesday, July 09, 2002 9:52 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: [SIGMETRICS] SV: [SIGMETRICS] Web citation issues

Dear colleagues, as you may be well aware of our department was one of the
first to engage into webometric analyses, in particular of web impact by
means of inlinks. Several others around have done deep analyses of
webometric nature to see if, for instance, there exists a correlation
between citation impact and web impact/no. of inlinks (Bar Ilan) or between
other indicators of recognition (e.g. RAE in the UK) and web impact
(Thelwall). Sometimes there exist such correlations, but the major problem
is that the web acts differently than scientific communication vehicles (no
real conventions of linking etc.). Also: one may not equalize a web page
with a scientific article. So Tom Wilson´s no. of hits (to web pages) does
not necessarily correspond at all to "scientific items citing other
scientific items" as in conventional citation analyses. Only the reading
lists found by Wilson may act like such "reference lists". Further:
Different search engines commonly perform differently on the same search
profile, resulting in biased counts; several large quality studies have been
done on "scientific" web page search results to observe the actual
proportion of scientific output (see e.g. Allen et al in Science in 1999.)
Results are appalling and pauvre - and highly dependent on the domain in
question. For instance, in politically hot scientific topics (like in the
environmental sc.-) there is a chaotic mix of scientific, semi-scientific,
pseudo, popular and, foremost, political opinion papers. Should all the
"published" pages count or only the peer reviewed ones - e.g. those
published by scientific institutions or referring to peer reviewed journals
or published in peer reviewed e-journals?
Additionally, there are several possible web impacts: by inlink counts; by
inlink counts and outlink counts; by web-based traditional
references/citations on all the open web - or only in e-journals - with or
without peer review.
Tom will of course also run into the problem similar to that of ISI: in the
citation databases no citations from books and non-ISI journals are counted;
on the web only the open weblinks (and citations/references) are possible to
count - not links/citations provided by pages/items on the hidden web (e.g.
all the Dialog or ISI databases or the journals in publisher archives or in
Digital Libraries.
Finally, on the web the nature of obsolescence of information is quite
different and not yet well understood - see e.g. Ronald Rousseau´s articles
in Cybermetrics or recent publications by Wolfgang Glanzel on the issue.
A recent review on some of the issues touched upon above is: Bjorneborn, L.
& Ingwersen, P.: Perspectives of webometrics. Scientometrics, 50(1): 65-82.

Many regards - Peter Ingwersen
********************************************
Peter Ingwersen, Professor, Ph.D.
Department of Information Studies
Royal School of Library and Information Science,
Birketinget 6, DK 2300 Copenhagen S - Denmark
Tel: +45 32 58 60 66; FAX: +45 32 84 02 01
http://www.db.dk/pi/ - e-mail: pi at db.dk
Visiting Professor (Docent), Dept. of Information Studies
Åbo University Akademi - Finland
********************************************

-----Oprindelig meddelelse-----
Fra: Quentin L. Burrell [mailto:quentinburrell at MANX.NET]
Sendt: 21. juni 2002 18:11
Til: SIGMETRICS at LISTSERV.UTK.EDU
Emne: Re: [SIGMETRICS] Web citation (fwd)

I am wholly in support of the SIGMETRICS site being one for discussion and
so was interested in Tom's original submission and now Gene's response. Here
comes my two penn'orth (Eng., coll., obs.?)

Tom's observations are interesting - to his comment on token citations I
would add (many cases of) self-citation - but I go along with Gene's
uneasiness on the current haphazard coverage of the web being adequate to
replace formal citation bases.

Gene's final remark that the "ultimate objective of universal
bibliographical control is to find it all in one place, displayed in a
fashion that is easily and quickly comprehended" surely requires some
response.

(i)I guess that "control" was a hasty first attempt and that "information"
is more in line with the intended meaning.

(ii) I would really like to see the phrase "freely available" inserted
somewhere in the remark. At the moment, unless you are the member of a
subscribing institution you don't have free access to this bibliographic
information, either to "boost your ego" or to measure your impact.Citation
analysis - like any othe form of data analysis - requires access to the
data.

Anyone else willing to chip in a cent or a yen or a euro or a ... ?

Quentin Burrell

-----Original Message-----
From: ASIS Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU]On Behalf Of Garfield, Eugene
Sent: 19 June 2002 18:27
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Web citation (fwd)

It is almost six weeks since Tom Wilson posted this message. Many others
forwarded copies to me expecting me to respond to his challenge.

It is always pleasant to learn that one's work has been mentioned on a
particular web site or that it is discussed in various courses. But while
they are newsworthy they havoc little real bearing on the use of citation
indexes to measure the impact of one's research.

When you are quite young anything helps boost the ego, but the bottom line
for the researcher is whether anyone has used his or her basic ideas in
ongoing research. Until that day of Nirvana arrives when everything will be
searchable on the web I am afraid web searching just won't be an adequate
substitute.

If you are working in the life sciences you can find many relevant citation
connections through such full text resources as HighWire Press, but that is
not yet complete nor is it presented in a form that is easily used for
citation analysis. That day may come. Steve Lawrence's project at NEC which
provides citation indexing in context for the computer science literature
illustrate what happens when you have only partial coverage.

The ultimate objective of universal bibliographical control is to find it
all in one place, displayed in a fashion that is easily and quickly
comprehended. Gene Garfield

When responding, please attach my original message
__________________________________________________
Eugene Garfield, PhD. email:  garfield at codex.cis.upenn.edu
home page: www.eugenegarfield.org
Tel: 215-243-2205 Fax 215-387-1266
President, The Scientist LLC. www.the-scientist.com
Chairman Emeritus, ISI www.isinet.com
Past President, American Society for Information Science and Technology
(ASIS&T) www.asis.org

-----Original Message-----
From: Gretchen Whitney [mailto:gwhitney at UTK.EDU]
Sent: Monday, May 06, 2002 10:10 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: [SIGMETRICS] Web citation (fwd)

---------- Forwarded message ----------
Date: Mon, 06 May 2002 12:25:22 +0100
From: Prof. Tom Wilson <t.d.wilson at SHEF.AC.UK>
To: JESSE at LISTSERV.UTK.EDU
Subject: Web citation

There have been a few mentions of Web citation searching possibly replacing
citation indexing in time and I wondered how many people are now, as a
matter of course, using counts of Web mentions in their cases for
appointment, tenure or promotion.

I looked at a couple of my own papers and counted the SSCI citations and
then searched for mentions of the papers on the Web - the results left me
wondering whether the reliance on citation indexing as a measure of
performance is now past its sell by date.

My most cited paper is "On user studies and information needs" (1981) - a
Web search (using Google) revealed 118 pages that listed the title. The
pages were reading lists, free electronic journals, and documents that would
never be covered by SSCI, such as reports from various agencies. SSCI
revealed, if I recall aright, 79 citations of the paper. The question is: is
the Web revealing impact more effectively than SSCI?  Citation in scholarly
papers takes a variety of forms and much citation is of a token variety - x
is cited because x is always cited. On the other hand citation on reading
lists implies some positive recommendation of the text, and mention in
policy documents and the like, implies (at least in some cases) that some
benefit has been found in the cited document.

It may also be that the use of Web citation would provide a more complete
measure - I discovered, much to my surprise, that a 1971 text of mine on
'chain indexing' is cited on one reading list and in the bibliography of a
document in German on classification. Greater international coverage is a
further benefit of using Web citation.

It strikes me that a move towards using Web citation as the measure of
performance would be rather more useful than the use of citation indexes.

No doubt others have looked at this issue - is any consensus emerging?

Tom Wilson

-----------------------------------
Professor T.D. Wilson, PhD
Publisher/Editor in Chief
Information Research
University of Sheffield
Sheffield S10 2TN
United Kingdom
Tel: +44-114-222-2642
E-mail: t.d.wilson at shef.ac.uk
Web site: http://InformationR.net/
-----------------------------------