Web citation issues

[Isidro F. Aguillo]

Dear all:

I arrive a bit late to this debate, but I think an important point is
missing in several of the messages. Most of them focus on the
comparision between links and citations and the reasons to cite and the
objects to be cited. Certainly both wordls are different and printed
literature and bibliographic references are superior for bibliometric
analysis.

But our traditional approach has several shortcommings as several
authours have already pointed. From the evaluation and policy purposes
point of view, one of the most important is that there are no enough
readily available information for a real sciencetechnoeconometrics. The
interlinked nature of the Web offer the possibility to discover hidden
relationships among different websites, including not only the bonds
that binds academic communities but those showing economic, industrial,
social or cultural relationships. And this is not easy with traditional
quantitative methods.

Going to the personal aspects, the presence on the Web of research
groups, professors or postgraduate students reflects a wider range of
activities than formal publication in refereed journals, such as
unpublished material, general public contributions, drafts for future
papers or book chapters, slides used in conference or seminar
presentations, support material for courses or even raw data. The
picture you can obtain from a researcher is therefore more complete.

Finally, as tax-payer citizen supporting public research activities I
request better access to the results obtained by scientists and in this
sense the Web reaches a wider audience than the paper based publications
like journals or books. In fact almost all the information published on
the Web can be recover by any Internet user worldwide.

Comments ...

Sloan, Bernie wrote:

> Peter,
>
> For the past two years or so I have been dabbling in the collection of
> citations and inlinks to several papers of mine. I use your definition of
> "inlinks": "links pointing to web pages" (Bjorneborn & Ingwersen). I've been
> using both publicly available Web search engines (Google and AllTheWeb), as
> well as some commercial subscription-only databases from providers (e.g.,
> ISI, the Gale Group, and EBSCO). I tried to be somewhat selective and did
> some brief analysis of the items I retrieved, e.g., I did not count inlinks
> from personal bookmark lists, or references to the papers from discussion
> group archives.
>
> I don't pretend that my papers are scientific papers, or that the citations
> and inlinks I have discovered are for "scientific items citing other
> scientific items." I did my best to eliminate any marginal entries, although
> "marginal" is definitely in the eye of the beholder here. I like to think of
> what I am doing as using the Web to discover the "influence of ideas". The
> exercise started with the Web of Science, and then broadened out to include
> the resources mentioned in the preceding paragraph. I don't think of the
> exercise as something that replaces ISI databases such as SCI, SSI, etc.,
> but rather something that supplements the information obtained from ISI.
>
> One of the most interesting findings is that this exercise uncovered an
> international influence that I would not have been aware of using only the
> ISI databases. More than 40% of the citations/inlinks were from non-US
> sources. Of the citations/inlinks from non-US sources, fully 60% were
> associated with countries with a primary language other than English.
>
> Right now I am working on putting together a paper reporting on my findings,
> discoveries and questions in more detail, sort of a report on a
> work-in-progress. The biggest question I am grappling with is "So what?" I
> have a detailed listing of citations/inlinks for several of my papers.
> Beyond satisfying personal and academic curiosity, what can these "personal
> citation indices" be used for? Using them for evaluation springs to mind,
> but of course "evaluation" requires comparison against some standard or
> control group. I'm not aware of too many people who have conducted this
> exercise at a personal level. I have something that seems impressive to me,
> in a relative sense (i.e., a more comprehensive overview of the influence of
> ideas than I might obtain from a citation index like SSI). But what can I
> use it for?
>
> Bernie Sloan
>
> -----Original Message-----
> From: Ingwersen Peter [mailto:PI at DB.DK]
> Sent: Tuesday, July 09, 2002 9:52 AM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: [SIGMETRICS] SV: [SIGMETRICS] Web citation issues
>
> Dear colleagues, as you may be well aware of our department was one of the
> first to engage into webometric analyses, in particular of web impact by
> means of inlinks. Several others around have done deep analyses of
> webometric nature to see if, for instance, there exists a correlation
> between citation impact and web impact/no. of inlinks (Bar Ilan) or between
> other indicators of recognition (e.g. RAE in the UK) and web impact
> (Thelwall). Sometimes there exist such correlations, but the major problem
> is that the web acts differently than scientific communication vehicles (no
> real conventions of linking etc.). Also: one may not equalize a web page
> with a scientific article. So Tom Wilson´s no. of hits (to web pages) does
> not necessarily correspond at all to "scientific items citing other
> scientific items" as in conventional citation analyses. Only the reading
> lists found by Wilson may act like such "reference lists". Further:
> Different search engines commonly perform differently on the same search
> profile, resulting in biased counts; several large quality studies have been
> done on "scientific" web page search results to observe the actual
> proportion of scientific output (see e.g. Allen et al in Science in 1999.)
> Results are appalling and pauvre - and highly dependent on the domain in
> question. For instance, in politically hot scientific topics (like in the
> environmental sc.-) there is a chaotic mix of scientific, semi-scientific,
> pseudo, popular and, foremost, political opinion papers. Should all the
> "published" pages count or only the peer reviewed ones - e.g. those
> published by scientific institutions or referring to peer reviewed journals
> or published in peer reviewed e-journals?
> Additionally, there are several possible web impacts: by inlink counts; by
> inlink counts and outlink counts; by web-based traditional
> references/citations on all the open web - or only in e-journals - with or
> without peer review.
> Tom will of course also run into the problem similar to that of ISI: in the
> citation databases no citations from books and non-ISI journals are counted;
> on the web only the open weblinks (and citations/references) are possible to
> count - not links/citations provided by pages/items on the hidden web (e.g.
> all the Dialog or ISI databases or the journals in publisher archives or in
> Digital Libraries.
> Finally, on the web the nature of obsolescence of information is quite
> different and not yet well understood - see e.g. Ronald Rousseau´s articles
> in Cybermetrics or recent publications by Wolfgang Glanzel on the issue.
> A recent review on some of the issues touched upon above is: Bjorneborn, L.
> & Ingwersen, P.: Perspectives of webometrics. Scientometrics, 50(1): 65-82.
>
> Many regards - Peter Ingwersen
> ********************************************
> Peter Ingwersen, Professor, Ph.D.
> Department of Information Studies
> Royal School of Library and Information Science,
> Birketinget 6, DK 2300 Copenhagen S - Denmark
> Tel: +45 32 58 60 66; FAX: +45 32 84 02 01
> http://www.db.dk/pi/ - e-mail: pi at db.dk
> Visiting Professor (Docent), Dept. of Information Studies
> Åbo University Akademi - Finland
> ********************************************
>
>
> -----Oprindelig meddelelse-----
> Fra: Quentin L. Burrell [mailto:quentinburrell at MANX.NET]
> Sendt: 21. juni 2002 18:11
> Til: SIGMETRICS at LISTSERV.UTK.EDU
> Emne: Re: [SIGMETRICS] Web citation (fwd)
>
>
> I am wholly in support of the SIGMETRICS site being one for discussion and
> so was interested in Tom's original submission and now Gene's response. Here
> comes my two penn'orth (Eng., coll., obs.?)
>
> Tom's observations are interesting - to his comment on token citations I
> would add (many cases of) self-citation - but I go along with Gene's
> uneasiness on the current haphazard coverage of the web being adequate to
> replace formal citation bases.
>
> Gene's final remark that the "ultimate objective of universal
> bibliographical control is to find it all in one place, displayed in a
> fashion that is easily and quickly comprehended" surely requires some
> response.
>
> (i)I guess that "control" was a hasty first attempt and that "information"
> is more in line with the intended meaning.
>
> (ii) I would really like to see the phrase "freely available" inserted
> somewhere in the remark. At the moment, unless you are the member of a
> subscribing institution you don't have free access to this bibliographic
> information, either to "boost your ego" or to measure your impact.Citation
> analysis - like any othe form of data analysis - requires access to the
> data.
>
> Anyone else willing to chip in a cent or a yen or a euro or a ... ?
>
> Quentin Burrell
>
>
> -----Original Message-----
> From: ASIS Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU]On Behalf Of Garfield, Eugene
> Sent: 19 June 2002 18:27
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] Web citation (fwd)
>
>
> It is almost six weeks since Tom Wilson posted this message. Many others
> forwarded copies to me expecting me to respond to his challenge.
>
> It is always pleasant to learn that one's work has been mentioned on a
> particular web site or that it is discussed in various courses. But while
> they are newsworthy they havoc little real bearing on the use of citation
> indexes to measure the impact of one's research.
>
> When you are quite young anything helps boost the ego, but the bottom line
> for the researcher is whether anyone has used his or her basic ideas in
> ongoing research. Until that day of Nirvana arrives when everything will be
> searchable on the web I am afraid web searching just won't be an adequate
> substitute.
>
> If you are working in the life sciences you can find many relevant citation
> connections through such full text resources as HighWire Press, but that is
> not yet complete nor is it presented in a form that is easily used for
> citation analysis. That day may come. Steve Lawrence's project at NEC which
> provides citation indexing in context for the computer science literature
> illustrate what happens when you have only partial coverage.
>
> The ultimate objective of universal bibliographical control is to find it
> all in one place, displayed in a fashion that is easily and quickly
> comprehended. Gene Garfield
>
> When responding, please attach my original message
> __________________________________________________
> Eugene Garfield, PhD. email:  garfield at codex.cis.upenn.edu
> home page: www.eugenegarfield.org
> Tel: 215-243-2205 Fax 215-387-1266
> President, The Scientist LLC. www.the-scientist.com
> Chairman Emeritus, ISI www.isinet.com
> Past President, American Society for Information Science and Technology
> (ASIS&T) www.asis.org
>
>
>
>
>
>
> -----Original Message-----
> From: Gretchen Whitney [mailto:gwhitney at UTK.EDU]
> Sent: Monday, May 06, 2002 10:10 AM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: [SIGMETRICS] Web citation (fwd)
>
>
> ---------- Forwarded message ----------
> Date: Mon, 06 May 2002 12:25:22 +0100
> From: Prof. Tom Wilson <t.d.wilson at SHEF.AC.UK>
> To: JESSE at LISTSERV.UTK.EDU
> Subject: Web citation
>
> There have been a few mentions of Web citation searching possibly replacing
> citation indexing in time and I wondered how many people are now, as a
> matter of course, using counts of Web mentions in their cases for
> appointment, tenure or promotion.
>
> I looked at a couple of my own papers and counted the SSCI citations and
> then searched for mentions of the papers on the Web - the results left me
> wondering whether the reliance on citation indexing as a measure of
> performance is now past its sell by date.
>
> My most cited paper is "On user studies and information needs" (1981) - a
> Web search (using Google) revealed 118 pages that listed the title. The
> pages were reading lists, free electronic journals, and documents that would
> never be covered by SSCI, such as reports from various agencies. SSCI
> revealed, if I recall aright, 79 citations of the paper. The question is: is
> the Web revealing impact more effectively than SSCI?  Citation in scholarly
> papers takes a variety of forms and much citation is of a token variety - x
> is cited because x is always cited. On the other hand citation on reading
> lists implies some positive recommendation of the text, and mention in
> policy documents and the like, implies (at least in some cases) that some
> benefit has been found in the cited document.
>
> It may also be that the use of Web citation would provide a more complete
> measure - I discovered, much to my surprise, that a 1971 text of mine on
> 'chain indexing' is cited on one reading list and in the bibliography of a
> document in German on classification. Greater international coverage is a
> further benefit of using Web citation.
>
> It strikes me that a move towards using Web citation as the measure of
> performance would be rather more useful than the use of citation indexes.
>
> No doubt others have looked at this issue - is any consensus emerging?
>
> Tom Wilson
>
> -----------------------------------
> Professor T.D. Wilson, PhD
> Publisher/Editor in Chief
> Information Research
> University of Sheffield
> Sheffield S10 2TN
> United Kingdom
> Tel: +44-114-222-2642
> E-mail: t.d.wilson at shef.ac.uk
> Web site: http://InformationR.net/
> -----------------------------------
>
>


--
--------------------------------------------------------
Isidro F. Aguillo       isidro at cindoc.csic.es
Laboratorio de Internet. CINDOC-CSIC
Joaquin Costa, 22       Ph. +34-91-5635482 ext. 313
28002 Madrid            Mb. +34-630-858997
ESPAÑA-SPAIN
        Fx. +34-91-5642644
Editor Cybermetrics     www.cindoc.csic.es/cybermetrics
--------------------------------------------------------