UK Research Assessment Exercise (RAE) review
harnad at ECS.SOTON.AC.UK
Fri Nov 22 22:28:27 EST 2002
On Fri, 22 Nov 2002, David Goodman wrote:
> I do not think that the comparison of the eventual value of the
> different specialties of scientific research can be judged at the
> time the research is being done.
"Can only be predicted with XX% reliability" is the statistically sound
way of putting it. And both XX and the time-span will vary (with time,
Assessors for research funding don't ask for 100% predictive accuracy. They
just want something like "Research/Researcher A is more likely than B"
(when funds are finite).
> That requires historical knowledge as well as scientometrics.
Yes, but hindsight is not a predictor (unless it picks out a predictive
pattern or index for the next time).
> This does imply a certain humility about the ability to use current
> knowledge as a valid basis for long term science policy.
I don't know about long-term science policy. The RAE just wants some
objective help in disbursing support for the next few years.
> Your second derivative technique, if the data are sufficiently
> accurate to support it, sounds like an exceeding nice way of measuring the
> potential short-term rise of a scientific field (or department). I would
> be reluctant to extrapolate very far into the future with such methods.
Extrapolate no further than your time-series correlations suggest you
have a statistical basis for extrapolating.
> For example, as judged by apparent current productivity, and its apparent
> valuation by the scientific world in general, scientometrics does not
> show very well. You and I know better, of course. :)
The time-line for the betting on scientometrics is still very
short, the field being new and its database growing. Its day is fast
coming, though, and open-access (along with the scientometric analzers
like http://citebase.eprints.org/ ) will help usher it in.
More information about the SIGMETRICS