Open Access Metrics: Use REF2014 to Validate Metrics for REF2020

[Stevan Harnad]

Steven Hill of HEFCE has posted “an overview of the work HEFCE are
currently commissioning which they are hoping will build a robust evidence
base for research assessment” in LSE Impact Blog 12(17) 2014 entitled Time
for REFlection: HEFCE look ahead to provide rounded evaluation of the REF
<http://blogs.lse.ac.uk/impactofsocialsciences/2014/12/17/time-for-reflection/>

Let me add a suggestion, updated for REF2014, that I have made before
(unheeded):

Scientometric predictors of research performance need to be validated by
showing that they have a high correlation with the external criterion they
are trying to predict. The UK Research Excellence Framework (REF) --
together with the growing movement toward making the full-texts of research
articles freely available on the web -- offer a unique opportunity to test
and validate a wealth of old and new scientometric predictors, through
multiple regression analysis: Publications, journal impact factors,
citations, co-citations, citation chronometrics (age, growth, latency to
peak, decay rate), hub/authority scores, h-index, prior funding, student
counts, co-authorship scores, endogamy/exogamy, textual proximity,
download/co-downloads and their chronometrics, tweets, tags, etc.) can all
be tested and validated jointly, discipline by discipline, against their
REF panel rankings in REF2014. The weights of each predictor can be
calibrated to maximize the joint correlation with the rankings. Open Access
Scientometrics will provide powerful new means of navigating, evaluating,
predicting and analyzing the growing Open Access database, as well as
powerful incentives for making it grow faster.

Harnad, S. (2009) Open Access Scientometrics and the UK Research Assessment
Exercise <http://eprints.ecs.soton.ac.uk/17142/>. *Scientometrics* 79 (1) Also
in *Proceedings of 11th Annual Meeting of the International Society for
Scientometrics and Informetrics* 11(1), pp. 27-33, Madrid, Spain.
Torres-Salinas, D. and Moed, H. F., Eds.  (2007)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20141217/1e0a9b67/attachment.html>