STI conference Leiden--Quality standards for evaluation indicators

[Yves Gingras]

Hello all

Here is an importnt document about good and bad indicators to add up to the
discussion on research evaluation:

http://www.scienceadvice.ca/en/assessments/completed/science-performance.asp
x

The whole report is free in pdf.

Best regards


Yves Gingras





Le 27/08/14 20:21, « Ismael Rafols » <ismaelrafols at gmail.com> a écrit :

> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> With apologies for cross-posting)
> 
> Dear all,
> to warm up forweek in the ST Indicators Conference in Leiden, let us share the
> topic of a debate:
> 
> Quality standards for evaluation indicators: Any chance for the dream to come
> true?
> Special session at the STI-ENID conference in Leiden, 3 September
> 2014, 16-17.30h 
> Organisers: Ismael Rafols (INGENIO & SPRU), Paul Wouters (CWTS, Leiden
> University), Sarah de Rijcke (CWTS, Leiden University)
> Location:  Aalmarkt-hall, Stadsgehoorzaal Leiden
> 
> There is a growing realization in the scientometrics community of the need to
> offer clearer guidance to users and further develop standards for professional
> use of bibliometrics in research evaluations. Indeed the STI-ENID Conference
> 2014 has the telling sub-title ŒContext Matters¹. This session continues from
> the 2013 ISSI and STI conferences in Vienna and Berlin, where full plenary
> sessions were convened on the need for standards in evaluative bibliometrics,
> and the ethical and policy implications of individual-level bibliometrics. The
> need to debate these issues has come to the forefront in light of reports that
> uses of certain easy-to-use metrics for evaluative purposes have become a
> routine part of academic life, despite misgivings within the profession itself
> about its validity. Very recently high-profile movements against certain
> metric indicators (e.g. the DORA declaration about the Journal Impact Factor)
> have brought possible misuses of metrics further to the center of attention.
> There may be a growing need for standards ­ also to promote for accountability
> of scientometricians as experts.
> 
> Indeed the relationship between scientometricians and end-users has been
> changing over the years due to factors like: 1. Increasing demands for
> bibliometric services in research management at various levels of aggregation,
> 2. New capacities and demands for performance information through the greater
> availability of new research technologies and their applications, and 3. The
> emergence of ³citizen bibliometrics² (i.e. bibliometrics carried out by
> non-expert end-users) due to larger availability of data and indicators. Some
> of these developments may result in new opportunities for research
> contributions and information-use, and may increase effectiveness of
> bibliometrics due to more advanced indicators and increased availability of
> data sets (including web data). Yet some innovations also risk bypassing the
> quality control mechanisms of fields like scientometrics and the standards
> they promote. The implications of this increasing scope and intensity of
> bibliometric practices requires a concerted response from scientometrics to
> produce more explicit guidelines and expert advice on good scientometric
> practices for specific evaluative practices such as recruitment, grant awards,
> institutional or national benchmarking. 
> 
> This special session will bring together scientometric experts,
> representatives of funding agencies, policy makers and opinion leaders on the
> role of metrics in research assessment to discuss the extent to which moving
> towards clearer, standardised guidelines over usage and consultancy can be
> achieved, both technically and strategically, and what the guidelines should
> look like concretely.
> 
> ---
> Background material:
> - Report on International workshop "Guidelines and good practices on
> quantitative assessments of research" (OST, Paris, 12 May 2014):
>  http://www.obs-ost.fr/fractivit%C3%A9s/workshop_international
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=http%3a%2f%2fwww.obs-ost.fr%2ffractivit%25C3%25A9s%2fworkshop_international
> > 
> - Blogposts Paul Wouters on previous debates at the ISSI and STI conferences
> in 2013, and on the DORA declaration:
> http://citationculture.wordpress.com/2013/07/29/bibliometrics-of-individual-re
> searchers/ 
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=http%3a%2f%2fcitationculture.wordpress.com%2f2013%2f07%2f29%2fbibliometrics
> -of-individual-researchers%2f>
> http://citationculture.wordpress.com/2013/10/03/bibliometrics-of-individual-re
> searchers-the-debate-in-berlin/
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=http%3a%2f%2fcitationculture.wordpress.com%2f2013%2f10%2f03%2fbibliometrics
> -of-individual-researchers-the-debate-in-berlin%2f>
> http://citationculture.wordpress.com/2013/05/23/dora-a-stimulus-for-a-new-eval
> uation-culture-in-science/
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=http%3a%2f%2fcitationculture.wordpress.com%2f2013%2f05%2f23%2fdora-a-stimul
> us-for-a-new-evaluation-culture-in-science%2f>
> - Information on the Higher Education Funding Council for England (HEFCE)
> "Independent review of the role of metrics in research assessment" + SPRU
> response
> http://citationculture.wordpress.com/2014/05/02/metrics-in-research-assessment
> -under-review/ 
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=http%3a%2f%2fcitationculture.wordpress.com%2f2014%2f05%2f02%2fmetrics-in-re
> search-assessment-under-review%2f>
> http://www.hefce.ac.uk/whatwedo/rsrch/howfundr/metrics/
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=http%3a%2f%2fwww.hefce.ac.uk%2fwhatwedo%2frsrch%2fhowfundr%2fmetrics%2f>
> https://www.sussex.ac.uk/webteam/gateway/file.php?name=spru-response-final.pdf
> &site=25 
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=https%3a%2f%2fwww.sussex.ac.uk%2fwebteam%2fgateway%2ffile.php%3fname%3dspru
> -response-final.pdf%26site%3d25>
> - Opinion article for JASIST by Sarah de Rijcke and Alex Rushforth "To
> intervene, or not to intervene; is that the question? On the role of
> scientometrics in research evaluation."
> https://citationculture.files.wordpress.com/2014/08/de-rijcke_rushforth_jasist
> _preprint2014.pdf
> <https://agenda.upv.es/owa/redir.aspx?C=55006d6295ec47ed88d25cb18366d006&U
> RL=https%3a%2f%2fcitationculture.files.wordpress.com%2f2014%2f08%2fde-rijcke_r
> ushforth_jasist_preprint2014.pdf>
> 
> 


Yves Gingras

Professeur 
Département d'histoire
Centre interuniversitaire de recherche
sur la science et la technologie (CIRST)
Chaire de recherche du Canada en histoire
et sociologie des sciences
Observatoire des sciences et des technologies (OST)
UQAM
C.P. 8888, Succ. Centre-Ville
Montréal, Québec
Canada, H3C 3P8

Tel: (514)-987-3000-7053
Fax: (514)-987-7726

http://www.chss.uqam.ca
http://www.cirst.uqam.ca
http://www.ost.uqam.ca

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20140828/adb446c7/attachment.html>