Is bibliometrics at danger?

[Yves Gingras]

Hello

It is true that the flawed indicators used in many rankings generate
criticisms of the use of citations in evaluation but one should also note
that the promotion of so-called "altmetrics" is often understood as being an
'alternative' to citations as if the latter were not an adequate measure of
impact. This term should be replaced by a better one like webindicators or
netindicators or anything saying clearly that those are not 'alternative'
but different measures adapted to a different mode of diffusion.


Best reagrds


Yves Gingras



Le 07/10/14 10:16, « Isidro F. Aguillo » <isidro.aguillo at CCHS.CSIC.ES> a
écrit :

> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> Dear Wouter,
> 
> You are right, I remember the problem. But it was solved in the next
> edition as you can expect once the issue has been identified. But the
> situation I described has been already reported for several years.
> However my main point is proliferation of comments like:
> 
> "The perils of ranking things based on citations"
> 
> as appeared today in Twitter. Bad bibliometrics by main citation
> database developer is affecting very negatively to our discipline.
> 
> Best,
> 
> 
> On 06/10/2014 23:24, Gerritsma, Wouter wrote:
>> Adminstrative info for SIGMETRICS (for example unsubscribe):
>> http://web.utk.edu/~gwhitney/sigmetrics.html
>> 
>> Isidoro
>> 
>> Years ago a similar issue played around the Leiden ranking as well, with a
>> German university coming out in the first place, based on a single, young,
>> well cited paper.
>> This might be one of the reasons CWTS put more emphasis on the %top10% most
>> cited papers in the new versions of the Leiden Ranking
>> 
>> Wouter
>> 
>> -----Original Message-----
>> From: ASIS&T Special Interest Group on Metrics
>> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Isidro F. Aguillo
>> Sent: maandag 6 oktober 2014 10:36
>> To: SIGMETRICS at LISTSERV.UTK.EDU
>> Subject: [SIGMETRICS] Is bibliometrics at danger?
>> 
>> Adminstrative info for SIGMETRICS (for example unsubscribe):
>> http://web.utk.edu/~gwhitney/sigmetrics.html
>> 
>> During our last conferences (Vienna, Berlin, Leiden) we discussed the
>> problems related to the uncontrolled usage of bibliometric techniques by
>> people without enough knowledge of the quality standards needed for research
>> assessment. In fact with the spread usage of "bad"
>> bibliometrics the discipline is starting to be viewed as irrelevant or
>> seriously flawed and biased. It is important to read carefully the now famous
>> DORA declaration that not only discourages the usage of the impact factor but
>> it is also attacking the whole citation analysis as the recommended
>> evaluation tool.
>> 
>> I already mentioned during the Vienna session that from a practical point of
>> view the success of certain rankings of Universities that use flawed citation
>> data is also a source of potential danger for the prestige of the discipline.
>> 
>> A few days ago the British magazine Times Higher Education (THE) published
>> the last edition of its very popular ranking of Universities.
>> Besides a reputation survey-based indicator they also collect citation data
>> (30% of the overall score) that during the last years have produced very
>> striking results. Among others, you can check in the current edition that
>> Federico Santa Maria Technical University, Chile has a larger score than
>> Harvard or Princeton, Tokyo Metropolitan University larger than Caltech or
>> Stanford, or Bogazici University, Turkey is performing better than Oxford or
>> Cambridge.
>> 
>> My point here is that data does not come from a THE journalist but, surprise,
>> directly from Thomson Reuters, as stated in their methodology
>> webpage: "this year, our data supplier Thomson Reuters examined more than 50
>> million citations to 6 million journal articles, published over five years.
>> The data are drawn from the 12,000 academic journals indexed by Thomson
>> Reuters' Web of Science database and include all indexed journals published
>> between 2008 and 2012. Citations to these papers made in the six years from
>> 2008 to 2013 are also collected".
>> 
>> You can find a very good analysis with tables in the blog of Richard
>> Holmes: http://rankingwatch.blogspot.com/
>> 
>> I know that Thomson Reuters is an independent private company, but I wonder
>> if our community as represented in this forum could ask for a strong action
>> regarding this unfortunate situation.
>> 
> 

Yves Gingras

Professeur 
Département d'histoire
Centre interuniversitaire de recherche
sur la science et la technologie (CIRST)
Chaire de recherche du Canada en histoire
et sociologie des sciences
Observatoire des sciences et des technologies (OST)
UQAM
C.P. 8888, Succ. Centre-Ville
Montréal, Québec
Canada, H3C 3P8

Tel: (514)-987-3000-7053
Fax: (514)-987-7726

http://www.chss.uqam.ca
http://www.cirst.uqam.ca
http://www.ost.uqam.ca