Universality of scholarly impact metrics

[Fil Menczer]

This new paper should be relevant to discussions on impact metrics and
their biases  (apologies for cross-posting):

Universality of scholarly impact metrics

Jasleen Kaur, Filippo Radicchi, Filippo Menczer
Journal of Informetrics 7 (4): 924–932, 2013
http://dx.doi.org/10.1016/j.joi.2013.09.002
Preprint: http://arxiv.org/abs/1305.6339

Abstract: Given the growing use of impact metrics in the evaluation of
scholars, journals, academic institutions, and even countries, there
is a critical need for means to compare scientific impact across
disciplinary boundaries. Unfortunately, citation-based metrics are
strongly biased by diverse field sizes and publication and citation
practices. As a result, we have witnessed an explosion in the number
of newly proposed metrics that claim to be "universal." However, there
is currently no way to objectively assess whether a normalized metric
can actually compensate for disciplinary bias. We introduce a new
method to assess the universality of any scholarly impact metric, and
apply it to evaluate a number of established metrics. We also define a
very simple new metric hs, which proves to be universal, thus allowing
to compare the impact of scholars across scientific disciplines. These
results move us closer to a formal methodology in the measure of
scholarly impact.

Filippo Menczer
Professor of Informatics and Computer Science
Director, Center for Complex Networks and Systems Research
Indiana University, Bloomington
http://cnets.indiana.edu/people/filippo-menczer