identification of review articles

[David Goodman]

The current Web of Science policy is at http://kbportal.thomson.com/display/2/searchDirect/index.aspx?searchstring=document+type&searchtype=2&searchby=keywords&Catid=&SubCatid=&att=&remotesite=&search=
from which I quote 
"Criteria for designating a record as Review in Web of Science
 
Articles and reviews are considered to be different document types . A review is a "special" type of article. A record is designated as a review in WOS if it has over 100 cited references and the word "review" is indicated at least twice in the original article (or, there is something in the original article that designates it a "review"). Two of the review criteria should be met for classification as a review. It cites more than 100 references, it appears in a review publication or a review section of a journal, the word review or overview appears in its title, the abstract states that it is a review or survey."




David Goodman, Ph.D., M.L.S.
previously:
Bibliographer and Research Librarian
Princeton University Library

dgoodman at princeton.edu


----- Original Message -----
From: Ludo Waltman <ludo at LUDOWALTMAN.NL>
Date: Friday, May 7, 2010 4:55 am
Subject: Re: [SIGMETRICS] identification of review articles
To: SIGMETRICS at LISTSERV.UTK.EDU

> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> Dear Linda,
> 
> Some time ago I performed an (unpublished) analysis of the accuracy 
> of  
> the identification of reviews in Web of Science. At least in some  
> fields (subject categories) the distinction between ordinary 
> articles  
> and reviews is inaccurate. Consider the field of management. The  
> attached Figure 1 shows the distribution of publications in the 
> field  
> of management based on their number of references. Ordinary 
> articles  
> are indicated in blue and reviews in red. As can be seen, in the 
> field  
> of management almost all publications with more than 100 references 
> 
> are classified as reviews, while almost no publications with less 
> than  
> 100 references are classified as reviews. It is of course extremely 
> 
> unlikely that this is a correct classification. One would expect 
> the  
> proportion of reviews to be a gradually increasing function of the  
> number of references. Instead, the figure shows a sudden increase 
> at  
> 100 references. Similar observations can be made for other fields,  
> although management seems to be a quite extreme case. Pharmacology 
> &  
> pharmacy is an example of a field with a much more gradually  
> increasing proportion of reviews (see the attached Figure 2). So in 
> 
> this field the distinction between ordinary articles and reviews 
> may  
> be more accurate.
> 
> Best regards,
> 
> Ludo Waltman
> 
> 
> ========================================================
> Ludo Waltman MSc
> Researcher
> 
> Centre for Science and Technology Studies
> Leiden University
> P.O. Box 905
> 2300 AX Leiden
> The Netherlands
> 
> Willem Einthoven Building, Room B5-35
> Tel:      +31 (0)71 527 5806
> Fax:      +31 (0)71 527 3911
> E-mail:   waltmanlr at cwts.leidenuniv.nl
> Homepage: www.ludowaltman.nl
> ========================================================
> 
> 
> Quoting Linda Butler <linda.butler at ANU.EDU.AU>:
> 
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > I'm hoping someone on the list may be able to help with this 
> query ...
> >
> > Until now, I have often used separate field-normalised benchmarks 
>  
> > for articles and reviews.  However some recent work I have   
> > undertaken has made me question the wisdom of this.  My   
> > understanding is that Scopus and WoS both classify a publication 
> as   
> > a 'review' if it contains more than 100 references. I hadn't 
> thought  
> >  too closely about this methodology until I recently came across  
> > some  articles that both Scopus and WoS have classified as 
> reviews,  
> > but  which appear to be standard research articles (though with 
> lots  
> > of  references).  I'm now beginning to wonder whether I should  
> > continue  to used separate benchmarks for articles and reviews.  
> If  
> > it is only  one or two papers that crop up in a macro level  
> > analysis, then I  won't be too concerned.  But if there is a  
> > question mark over the  accuracy of this method for identifying  
> > reviews, and the problem is  more common than, then I will need 
> to  
> > rethink my methodology.
> >
> > Does anyone know of any empirical studies that have examined the  
> 
> > accuracy of this method for classifying a publication as a review?
> >
> > Or even if you don't know of any studies, have you come across   
> > similar concerns in any analyses you have undertaken?
> >
> > with thanks
> > Linda Butler
>