On Metrics and Metaphysics

[Stevan Harnad]

---------- Forwarded message ----------
From: Leslie Carr <lac -- ecs.soton.ac.uk>
Date: Wed, Oct 22, 2008 at 7:03 AM
Subject: Re: On Metrics and Metaphysics
To: AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM -- listserver.sigmaxi.org


On 21 Oct 2008, at 18:23, J.F.Rowland wrote:

> Stevan - You misunderstood Heather's point.  She didn't say the
> researcher - the author of the current research article in question -
> was little-known.
> She said the literary author that (s)he was studying was little-known.

> Therefore, not many researchers will be interested in that literary
> author, so not many people will cite the article, however good it is.

I think that the arguments that Heather put forward are not
fundamentally directed at metrics per se. They are arguments about the
distinction between research impact and research importance; it is the
researchers, the societies, the funding councils and governments who
need to answer these policy questions.

> There is a real and valid point in Heather's message, and simply
> saying 'use other metrics' is vague, to say the least.

Yes, it is, isn't it. When someone has a more concrete idea of what we
are measuring (quality? excellence? importance? impact?) then
doubtless we can be make a reasonable attempt to be more specific.

>  Please specify what metrics might be used to provide a valid
> quality measure to the work of researchers who study minority
> subjects which will excite interest, and therefore usage,
> and citations, from only a few people worldwide.

I'll take my turn in prolonging the confusion by remarking that
bibliographic items are only one kind of evidence that can be observed
and measured. Everyone in the UK is familiar with the RAE's "measures
of esteem" which supplemented the bibliographic submissions. Invited
lectures, committee memberships, journal editorships and the like are
all leaving auditable trails of evidence on the web which we can
measure and use to moderate citation-only statistics.
--
les Carr