On Metrics and Metaphysics

[Stevan Harnad]

On Wed, Oct 22, 2008 at 2:55 AM, Loet Leydesdorff <loet at leydesdorff.net> wrote:

>> SH: Fair question. Here is a quick and dirty solution, by way of
>> illustration. (Many more are possible along analogous lines.)
>>
>> (1) Compare like with like. So we are only looking at literary
>> studies. The subject matter can be narrowed down by Boolean means,
>> including taxonomic descriptors. No comparing Shakespeare scholarship
>> works with molecular biology.
>
> LL: Against any theoretical objection, one can always argue that it is possible
> to do things "quick and dirty". In my opinion, this approach can easily
> damage standards in this field.
>
> The delineation of fields and subfields, for example, cannot easily be
> discarded, particularly on the web. In other words, standards for the
> normalization are never to be taken lightly.

The question was: How can one create a metric that will detect the
impact of piece of work on a low-volume topic, such as a literary
study of a little-studied author.

My reply was to give a "quick and dirty" approximation to show that it
is possible in principle (normalize by volume). That does not mean
that the quick and dirty approximation is the sole or optimal
solution, just that a solution is possible.

{Les Carr has since also made the point that metrics cannot be
expected to be oracular: (LC: "Perhaps I might be permitted to throw
the ball back in your court. How
would *you* [or anyone] know that a paper in the (narrow but
important) field has excited the interest of anyone worldwide? Or even
excited the interest of "the right people"? Once you can answer that,
to the satisfaction of the author and their community, then Stevan
(for you challenged him in particular) might be able to devise a
metric for measuring it. Or, indirectly, devise a test for whether a
proposed battery of metrics will act as a reasonable proxy for the
judgement of experts in the field."}

Faut pas se faire plus royaliste que le roi...

Stevan Harnad