On Metrics and Metaphysics

[David E. Wojick]

Speaking metaphysically, I find the following problematic -- "The point is  that
metrics need to be plural, diverse, and validated and weighted by  field and
subfield."

I would argue that the concepts of field and subfield are too vague and unscientific to be the basis for objective metrics. Science is an ever changing body of inquiry, with personal, methodological and subject matter clusters that change with scale and over time, sometimes very rapidly. No two papers are about exactly the same thing. Each paper is related to its neighbors in multiple ways. Each object of study can be looked at in multiple ways.

Thus field and subfield are always artificial divisions of convenience imposed on an unbroken web of belief and inquiry. Moreover, these divisions can often be usefully made in multiple ways. If the science of science depends on these artificial divisions as a basis for measurement then we are in serious trouble.

David Wojick
DOE OSTI

>
>On 22-Oct-08, at 7:33 AM, Loet Leydesdorff wrote:
>
>> It seems to me that the expectation of the citation frequency is  among other
>> things a function of the local density of the citation network. A  problem,
>> however, remains how to define the locale: a journal, a theme, a  patent
>> class? "Quick and dirty" skips these problems, in my opinion. I  agree that
>> it may be pragmatical and shows that a solution is possible in  principle.
>
>With open access, it is no longer univariate (i.e., not just citation  counts) and
>it is definitely no longer journal-centric (author and article  metrics, not journal
>JIFs, though journal JIFs can be among the metrics used). The point is  that
>metrics need to be plural, diverse, and validated and weighted by  field and
>subfield.
>
>To repeat: The "quick and dirty" example I gave was not meant to be  used,
>but to show that solutions (many solutions) are possible in principle,  and
>that their main features are that they are (1) multivariate, (2) field  or even
>subfield-based, (3) require prior (joint) validation, field by field,  against an
>already validated or face-valid criterion (such as peer evaluation),  and, most
> important, they are (4) conditional on the provision of a full Open  Access
>database on which to base them -- a condition that does not yet exist,  but
>one for which we are now fighting (using the potential of multiple  Open Access
>metrics as an incentive).
>
>> The problem seems to me in the inference from aggregated citing  behavior to
>> an expectation of being cited. The analyst transposed the
>> citation-transaction matrix (Wouters, 1999).
>
>The matrix I have in mind is not a citation matrix, but a matrix  consisting of a
>rich and diverse set of metrics, including downloads, chronometrics
>(growth/decay of citations, downloads), co-citation metrics, co- authorship
>metrics, funding metrics, student metrics. patent metrics, link metrics,
>hub/authority metrics, endogamy/exogamy metrics, years of publication,
>total publications, tag metrics, comment metrics, semiometrics, and  more --
>all these harvested from the Open Access Research Web, once all articles
>are OA, citation-linked, and download-metered.
>
>The difference between this plurimetric world and the world of  univariate
>citations will be like the difference between night and day. But we  are still in
>the night...
>
>Stevan Harnad

-- 

"David E. Wojick, PhD" <WojickD at osti.gov>
Senior Consultant for Innovation
Office of Scientific and Technical Information
US Department of Energy
http://www.osti.gov/innovation/
391 Flickertail Lane, Star Tannery, VA 22654 USA
540-858-3136

http://www.bydesign.com/powervision/resume.html provides my bio and past client list. 
http://www.bydesign.com/powervision/Mathematics_Philosophy_Science/ presents some of my own research on information structure and dynamics.