AW: [SIGMETRICS] Qualitative citation analysis?

[Bornmann Lutz]

Dear Bernie,
 
Our paper entitled "What do citation counts measure?" might be of interest to you. It is a review of studies on citing behavior that is accepted for publication in the Journal of Documentation. You can download the paper from my personal homepage: www.lutz-bornmann.de/Publications.htm
 
Kind regards
Lutz
 
 
-----------------------------------------------------------------------------
Dr. Lutz Bornmann
ETH Zurich, D-GESS
Professorship for Social Psychology and Research on Higher Education
Zaehringerstr. 24 / ZAE
CH-8092 Zurich
Phone: 0041 44 632 48 25
Fax: 0041 44 632 12 83
http://www.psh.ethz.ch/index_EN <http://www.psh.ethz.ch/> 
bornmann at gess.ethz.ch
 
Download of publications: www.lutz-bornmann.de/Publications.htm

________________________________

Von: ASIS&T Special Interest Group on Metrics im Auftrag von Karen Medina
Gesendet: Di 20.11.2007 21:05
An: SIGMETRICS at LISTSERV.UTK.EDU
Betreff: Re: [SIGMETRICS] Qualitative citation analysis?




Bernie wrote:
>I am interested in hearing about relatively
>   recent qualitative citation analysis.

Dear Bernie,

One paper you might be interested in is a study of how Gerard Salton was mis-cited and mis-interpreted for years (Dubin, (2004). The Most Influential Paper Gerard Salton Never Wrote. Library Trends. http://www.ideals.uiuc.edu/bitstream/2142/1697/2/Dubin748764.pdf
)

It is an interesting question you ask, and I think it has a complex answer. As a new person to bibliometrics, let me try to join the conversation early so that I can be corrected.

First, I think I'll ask what it is that qualitative analysis adds to citation studies? The greatest thing I think it adds is context that can be used for document retrieval systems (I think several people have mentioned this, Garfield for instance). Today, if you look at ISI, CiteSeer, and many Indexing and Abstracting software, you will notice that more and more of them are retrieving the context of the citation and presenting it to the user. So, in a way, systems are allowing the user to do the qualitative analyses that interest you.

I'll take on your point that the negative citations are different than positive citations. If what we are measuring is a impact on a field, then an author or paper that is negatively cited is still impacting the field. We have learned that negative citations tend to take more text space in the citing document. Perhaps we have more to learn about negative citations, but we have to critically evaluate what we want to measure.

Henry Small's 1978 paper, Cited Documents as Concept Symbols, summed up what had been happening with qualititative studies -- that some people were interested in the motivation for citing, while others were wanting to give some value judgement to the citation (calling some citations perfunctory others organic).

Motivation is really hard to judge. Some thought that an outsider was a better judge -- more objective. Some thought that an expert in the field was a better judge of motivation. But it seemed that each study developed a different classification scheme. Personally, I think the motivation behind citation behavior can best be judged by the author(s) of the citing document. Self-citation and cronie citations are not as wide-spread as some poople thought, but the scientific community as a whole is aware of how the practice of such citations could inflate prestigue temporarily.

You mention that quantitative studies treat all citations as equal. Well, if we are measuring impact on a field or prestigue, to some degree, a citation is a citation is a citation. 

Citation context does have a lot of potential, but it takes a lot of work to analyze well. Systems are making it easier, and there are papers out there that are reporting on it. But as systems make qualitative studies easier, perhaps, they are decreasing the need for us to do such studies. They are already implementing what we would be proving.

-karen medina
student

---- Original message ----
>Date: Tue, 20 Nov 2007 08:25:01 -0800
>From: "B.G. Sloan" <bgsloan2 at YAHOO.COM> 
>Subject: [SIGMETRICS] Qualitative citation analysis? 
>To: SIGMETRICS at LISTSERV.UTK.EDU
>
  
>   A discussion on the liblicense list reminded me of
>   something I asked about a couple of years ago in
>   another forum...just curious if anyone on SIGMETRICS
>   can point to some recent relevant studies...
>   Most of the citation analysis studies I see nowadays
>   involve quantitative analyses for the most part.
>   Just wondering if many people are into studying
>   citations from a qualitative standpoint? For
>   example, in a lot of studies a citation is a
>   citation is a citation, with little concern for how
>   a given paper was cited qualitatively within the
>   context of the citing paper. For example, an author
>   could cite a paper very positively, or the citation
>   could be pretty much value-neutral, or the citation
>   could be negative. But in a quantitative analysis
>   these various types of citations pretty much all
>   carry the same weight.
>   
>   When I looked into this several years ago, a number
>   of people alerted me to some qualitative citation
>   studies. The interesting thing is that most of these
>   studies were maybe 20 years old, at least. It almost
>   seemed like people got away from doing qualitative
>   citation analyses as it got easier to do
>   quantitative analyses, i.e., as databases such as
>   the ISI indices became available in electronic form.
>   
>   Anyway, I am interested in hearing about relatively
>   recent qualitative citation analysis.
>   
>   Thanks,
>   
>   Bernie Sloan
>
>
>
>     ------------------------------------------------
>
>   Get easy, one-click access to your favorites. Make
>   Yahoo! your homepage.