Normalization of peer-evaluation measures of group research quality across academic disciplines

[Eugene Garfield]

Normalization of peer-evaluation measures of group research quality across 
academic disciplines

Author(s): Kenna, R (Kenna, Ralph); Berche, B (Berche, Bertrand)
Source: RESEARCH EVALUATION  Volume: 20  Issue: 2  Pages: 107-116  DOI: 
10.3152/095820211X12941371876625  Published: JUN 2011  

Abstract: Peer-evaluation-based measures of group research quality such as 
the UK's Research Assessment Exercise (RAE), which do not employ bibliometric 
analyses, cannot directly avail of such methods to normalize research impact 
across disciplines. This is seen as a conspicuous flaw of such exercises and 
calls have been made to find a remedy. Here a simple, systematic solution is 
proposed based upon a mathematical model for the relationship between 
research quality and group quantity. This model manifests both the Matthew 
effect and a phenomenon akin to the Ringelmann effect and reveals the 
existence of two critical masses for each academic discipline: a lower value, 
below which groups are vulnerable, and an upper value beyond which the 
dependency of quality on quantity reduces and plateaus appear when the 
critical masses are large. A possible normalization procedure is then to pitch 
these plateaus at similar levels. We examine the consequences of this 
procedure at RAE for a multitude of academic disciplines, corresponding to a 
range of critical masses.

Language: English
Document Type: Article
KeyWords Plus: BIBLIOMETRIC STATISTICAL PROPERTIES; SCIENCE SYSTEM; 
SCALING RULES; GROUP-SIZE; UNIVERSITIES; PERFORMANCE; INDICATORS; 
CHEMISTRY; IMPACT

Addresses: [Kenna, R] Coventry Univ, Appl Math Res Ctr, Coventry CV1 5FB, W 
Midlands, England
Reprint Address: Kenna, R (reprint author), Coventry Univ, Appl Math Res Ctr, 
Coventry CV1 5FB, W Midlands, England
IDS Number: 789SS
ISSN: 0958-2029
PDF: http://arxiv.org/PS_cache/arxiv/pdf/1006/1006.3863v2.pdf