[Sigmetrics] Citizen bibliometrics - an alternative take

Mon Sep 19 12:04:20 EDT 2016

The overview of Yves’ book notes that

*"Bibliometrics is hailed as an “objective” measure of research quality, a
quantitative measure more useful than “subjective” and intuitive evaluation
methods such as peer review that have been used since scientific papers
were first published in the seventeenth century.”*

and I presume he will be criticizing this trend.

But I don’t understand why the implied issue is “objective bibliometrics"
vs. "subjective peer review.”

First, the bibliometrics are mostly applied to peer-reviewed research. So
there is both; the bibliometrics are a complement or supplement, not a
substitute for peer review(except for a few wacky proposals)

Second, peer review is not entirely subjective: referees have to give
reasons and evidence for their judgments.

(Note that I am defending neither blinkered reliance on bibliometrics nor
excess confidence in the reliability or rigour of peer review.)

Stevan Harnad

On Mon, Sep 19, 2016 at 10:19 AM, Gingras, Yves <gingras.yves at uqam.ca>
wrote:

> Oups, here is the link:
>
> https://mitpress.mit.edu/books/bibliometrics-and-research-evaluation
>
> Best regards
>
> Yves Gingras
>
> De : SIGMETRICS <sigmetrics-bounces at asis.org> on behalf of Björn
> Hammarfelt <bjorn.hammarfelt at hb.se>
> Date : lundi 19 septembre 2016 02:47
> À : "sigmetrics at mail.asis.org" <sigmetrics at mail.asis.org>
> Objet : [Sigmetrics] Citizen bibliometrics - an alternative take
>
> In view of the interesting and important paper on citizen bibliometrics by
> Leydesdorff, Wouters and Bornman we wish to share our own modest take on
> this concept. While our account can, and should, be seen as complimentary
> to the overview provided by the authors above, we also add to the arguments
> made. In particular our study emphasizes that 'citizien bibliometricans'
> (in our case researchers using bibliometrics in assessment reports
> regarding candidates for academic positions) actually modify, construct and
> deconstruct indicators; they are not passive end-users. Moreover, it's not
> uncommon that they show to be quite knowledgeable about the strength and
> weaknesses of particular indicators.
>
> *Judging merits in the age of the h-index: Citizen bibliometrics in
> biomedicine and economics*
>
> Bjorn Hammarfelt, Alexander D. Rushforth
>
>
> *Abstract*: The number of publications has been a fundamental merit in
> the competition for academic positions since the late 18th century. Today,
> the simple counting of publications has been supplemented with a whole
> range of bibliometric measures, which supposedly not only measures the
> volume of research but also its impact. In this study, we investigate how
> bibliometrics are used for evaluating the impact and quality of
> publications in two specific settings: biomedicine and economics. Our study
> exposes the extent and type of metrics used in external evaluations of
> candidates for academic positions at Swedish universities. Moreover, we
> show how different bibliometric indicators, both explicitly and implicitly,
> are employed to value and rank candidates. Our findings contribute to a
> further understanding of bibliometric indicators as judgment devices that
> are employed in evaluating individuals and their published works within
> specific fields. We also show how expertise in using bibliometrics for
> evaluative purposes is negotiated at the interface between domain knowledge
> and skills in using indicators. In line with these results we propose that
> the use of metrics in this context is best described as a form of citizen
> bibliometrics - an underspecified term which we build upon in the paper.
>
> pre-print available at: http://arxiv.org/abs/1609.04931
>
> On 16/09/16 08:24, Loet Leydesdorff wrote:
>
>
>
> *Professional and Citizen Bibliometrics: Complementarities and
> ambivalences in the development and use of indicators
> <http://arxiv.org/abs/1609.04793> *
>
> Loet Leydesdorff, Paul Wouters, and Lutz Bornmann
>
>
>
> *Abstract:* Bibliometric indicators such as journal impact factors,
> h-indices, and total citation counts are algorithmic artifacts that can be
> used in research evaluation and management. These artifacts have no meaning
> by themselves, but receive their meaning from attributions in institutional
> practices. We distinguish four main stakeholders in these practices: (1)
> producers of bibliometric data and indicators; (2) bibliometricians who
> develop and test indicators; (3) research managers who apply the
> indicators; and (4) the scientists being evaluated with potentially
> competing career interests. These different positions may lead to different
> and sometimes conflicting perspectives on the meaning and value of the
> indicators. The indicators can thus be considered as boundary objects which
> are socially constructed in translations among these perspectives. This
> paper proposes an analytical clarification by listing an informed set of
> (sometimes unsolved) problems in bibliometrics which can also shed light on
> the tension between simple but invalid indicators that are widely used
> (e.g., the h-index) and more sophisticated indicators that are not used or
> cannot be used in evaluation practices because they are not transparent for
> users, cannot be calculated, or are difficult to interpret.
>
>
>
> Preprint available at http://arxiv.org/abs/1609.04793
>
> ** apologies for cross-postings.
>
>
> ------------------------------
>
> Loet Leydesdorff
>
> Professor, University of Amsterdam
> Amsterdam School of Communication Research (ASCoR)
>
> loet at leydesdorff.net ; http://www.leydesdorff.net/
> Associate Faculty, SPRU, <http://www.sussex.ac.uk/spru/>University of
> Sussex;
>
> Guest Professor Zhejiang Univ. <http://www.zju.edu.cn/english/>,
> Hangzhou; Visiting Professor, ISTIC,
> <http://www.istic.ac.cn/Eng/brief_en.html>Beijing;
>
> Visiting Professor, Birkbeck <http://www.bbk.ac.uk/>, University of
> London;
>
> http://scholar.google.com/citations?user=ych9gNYAAAAJ&hl=en
>
>
>
>
> _______________________________________________
> SIGMETRICS mailing listSIGMETRICS at mail.asis.orghttp://mail.asis.org/mailman/listinfo/sigmetrics
>
>
> --
> Björn Hammarfelt, PhD
> Lektor / Senior lecturer
> Bibliotekshögskolan / Swedish School of Library and Information Science
> Högskolan i Borås / University of Borås
> Visiting scholar at the Centre for Science and Technology Studies (CWTS) Leiden University
> E- mail:bjorn.hammarfelt at hb.se; b.m.s.hammarfelt at cwts.leidenuniv.nl
> Work: 033-435 40 00
> Cell: (0046)-737277921http://www.hb.se/Forskning/Forskare/Hammarfelt-Bjorn/www.bibliometri.net
>
>
> _______________________________________________
> SIGMETRICS mailing list
> SIGMETRICS at mail.asis.org
> http://mail.asis.org/mailman/listinfo/sigmetrics
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20160919/4f4c7356/attachment-0001.html>