Universality of scholarly impact metrics

Fil Menczer fil at INDIANA.EDU
Tue Oct 22 13:16:56 EDT 2013

This new paper should be relevant to discussions on impact metrics and
their biases  (apologies for cross-posting):

Universality of scholarly impact metrics

Jasleen Kaur, Filippo Radicchi, Filippo Menczer
Journal of Informetrics 7 (4): 924–932, 2013
Preprint: http://arxiv.org/abs/1305.6339

Abstract: Given the growing use of impact metrics in the evaluation of
scholars, journals, academic institutions, and even countries, there
is a critical need for means to compare scientific impact across
disciplinary boundaries. Unfortunately, citation-based metrics are
strongly biased by diverse field sizes and publication and citation
practices. As a result, we have witnessed an explosion in the number
of newly proposed metrics that claim to be "universal." However, there
is currently no way to objectively assess whether a normalized metric
can actually compensate for disciplinary bias. We introduce a new
method to assess the universality of any scholarly impact metric, and
apply it to evaluate a number of established metrics. We also define a
very simple new metric hs, which proves to be universal, thus allowing
to compare the impact of scholars across scientific disciplines. These
results move us closer to a formal methodology in the measure of
scholarly impact.

Filippo Menczer
Professor of Informatics and Computer Science
Director, Center for Complex Networks and Systems Research
Indiana University, Bloomington

More information about the SIGMETRICS mailing list