Let 1000 RAE Metric Flowers Bloom: Avoid Matthew Effect as Self-Fulfilling Prophecy

Wed Jun 21 10:09:46 EDT 2006

I wouldn't worry too much about the pedigree of the quote/cliche. It
has pretty much become generic now. (I had forgotten or perhaps never
knew its provenance!).

The sentiment was obviously bogus in Mao's case. He had no intention
of letting all voices be heard. In the case of OA metrics, in
contrast, we are not even talking about freedom to voice opinions, we
are talking about making published research freely accessible online,
along with all the objective metrics that that will generate. The
rest of the terms of the analogy simply do not hold. (One must make
too much of the irrelevant details of a metaphor, nor be too
superstitious...)

Stevan Harnad

On 21-Jun-06, at 9:23 AM, Stephen J Bensman wrote:
>
> That's not a good saying to launch your RAE metric reform.  Below
> is the
> origins of your campaign slogan.  You will note that it is a quote
> from
> Chairman Mao.    If it follows its model, it can lead to the forced
> rustication of numerous British academics.
>
> SB
>
>
>  1956-7 Hundred Flowers and Anti-rightist Campaigns. As part of the
> effort to encourage the participation of
>  intellectuals in the new regime, cultural and intellectual figures
> were encouraged to speak their minds on the state
>  of CCP rule and programs. Mao personally took the lead in the
> movement, which was launched under the classical
>  slogan "Let a hundred flowers bloom, let the hundred schools of
> thought contend". At first the party's repeated
>  invitation to air constructive views freely and openly was met
> with caution. By mid-1957, however, the movement
>  unexpectedly mounted, bringing denunciation and criticism against
> the party in general and the excesses of its
>  cadres in particular. Startled and embarrassed, leaders turned on
> the critics as "bourgeois rightists" and launched
>  the Anti-Rightist Campaign, resulting in a year-long purge of
> party members, sending officials "down" to the
>  countryside (xiafang), and persecuting intellectuals who spoke out
> against the party.
>
>
>
>
>
>
>
>
> Stevan Harnad <harnad at ECS.SOTON.AC.UK>@LISTSERV.UTK.EDU> on 06/21/2006
> 06:10:32 AM
>
> Please respond to ASIS&T Special Interest Group on Metrics
>        <SIGMETRICS at LISTSERV.UTK.EDU>
>
> Sent by:    ASIS&T Special Interest Group on Metrics
>        <SIGMETRICS at LISTSERV.UTK.EDU>
>
>
> To:    SIGMETRICS at LISTSERV.UTK.EDU
> cc:     (bcc: Stephen J Bensman/notsjb/LSU)
>
> Subject:    [SIGMETRICS] Let 1000 RAE Metric Flowers Bloom: Avoid
> Matthew
>        Effect as Self-Fulfilling Prophecy
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Comment on:
>
>     UK plans research funding overhaul
>     by Stephen Pinfield
>     The Scientist, Wednesday 21 June, 2006
>     http://www.the-scientist.com/news/display/23683/
>
> ----------------------------------------------------------------------
> -
>     Let 1000 RAE Metric Flowers Bloom:
>     Avoid Matthew Effect as Self-Fulfilling Prophecy
>
>     by Stevan Harnad
>
> The conversion of the UK Research Assessment Exercise (RAE) from the
> present costly, wasteful exercise to time-saving and cost-efficient
> metrics is welcome and overdue, but the worrying thing is that
> the RAE planners currently seem to be focused on just one metric --
> prior research funding -- instead of the full and rich spectrum of new
> (and old) metrics that will become available in an Open Access world,
> with all the research performance data digitally available online
> for analysis and use.
>
> Mechanically basing the future RAE rankings exclusively on prior
> funding
> would just generate a Matthew Effect (making the rich richer and the
> poor poorer), a self-fulfilling prophecy that is simply equivalent to
> increasing the amount given to those who were previously funded (and
> scrapping the RAE altogether, as a separate, semi-independent
> performance
> evaluator and funding source).
>
> What the RAE *should* be planning to do is to look at weighted
> combinations of all available research performance metrics --
> including
> the many that are correlated, but not so tightly correlated, with
> prior
> RAE rankings, such as author/article/book citation counts, article
> download counts, co-citations (co-cited with and co-cited by, weighted
> with the citation weight of the co-citer/co-citee), endogamy/exogamy
> metrics (citations by self or collaborators versus others, within
> and across disciplines), hub/authority counts (in-cites and out-cites,
> weighted recursively by the citation's own in-cite and out-cite
> counts),
> download and citation growth rates, semantic-web correlates, etc.
>
> It would be both arbitrary and absurd to blunt the sensitivity, power,
> predictivity and validity of metrics a priori by biasing them
> toward the
> prior-funding counts metric alone, which should just be one out of
> a full
> battery of weighted metrics, adjusted to each discipline and validated
> against one another (and against human judgment too).
>
>     Shadbolt, N., Brody, T., Carr, L. and Harnad, S. (2006) The Open
>     Research Web: A Preview of the Optimal and the Inevitable, in
> Jacobs,
>     N., Eds. Open Access: Key Strategic, Technical and Economic
> Aspects,
>     chapter 21. Chandos.
>     http://eprints.ecs.soton.ac.uk/12369/
>
> ----------------------------------------------------------------------
> --
> Stevan Harnad
> AMERICAN SCIENTIST OPEN ACCESS FORUM:
> A complete Hypermail archive of the ongoing discussion of providing
> open access to the peer-reviewed research literature online
> (1998-2005)
> is available at:
> http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/
>         To join or leave the Forum or change your subscription
> address:
> http://amsci-forum.amsci.org/archives/American-Scientist-Open-
> Access-Forum.html
>
>         Post discussion to:
>         american-scientist-open-access-forum at amsci.org
>
> UNIVERSITIES: If you have adopted or plan to adopt an institutional
> policy of providing Open Access to your own research article output,
> please describe your policy at:
>         http://www.eprints.org/signup/sign.php
>
> UNIFIED DUAL OPEN-ACCESS-PROVISION POLICY:
>     BOAI-1 ("green"): Publish your article in a suitable toll-access
> journal
>             http://romeo.eprints.org/
> OR
>     BOAI-2 ("gold"): Publish your article in a open-access journal
> if/when
>             a suitable one exists.
>             http://www.doaj.org/
> AND
>     in BOTH cases self-archive a supplementary version of your article
>             in your institutional repository.
>             http://www.eprints.org/self-faq/
>             http://archives.eprints.org/
>             http://openaccess.eprints.org/