New citation indicators needed to measure research performance
johannes.stegmann at FU-BERLIN.DE
Thu Feb 21 05:42:58 EST 2013
Dear Lutz, dear Loet.
@Lutz: Of course, I agree with you that whatever evaluation technique is
applied, it should be transparent.
If journals are the reference one could add the information of their
within their subject category in the respective publication year.
themselves might be questionable in terms of compositions, coverage,
To use the papers of the same journal in which the paper to be compared
has been published as reference, seems (to me) intuitively to be more
In addition: if you analyse a research institute then you have normally
to deal with a mixture of low and high value journals which makes the
normalisation per se. It would be really interesting to find out whether
Figure 1 (comparison of institute 1 and institute 2) in your EMBO-Report
paper would look like more or less the same or quite different if you
apply percentile ranking using the journals involved as reference base.
@Loet: Data collection is boring because for each paper you want to rank
you have to download the respective journal/publication year pair. For
our DRFZ study (see Pudovkin et al. and Kretschmer et al, Scientometrics
Vol. 93 (2012), pp 3-16 and 17-30) we had to deal with over 300 papers
published (by the DRFZ) in nearly 100 journals. To determine the PRI of
each paper we had to download more than 260 journal/publication year
pairs which serve as reference for the ranking. The analysis with
respect to the determination of the position of each of the DRFZ papers
within its journal/year reference was/is easy using PERL and R scripts
for extraction of document id's and TIMES CITED numbers and appropriate
This kind of analysis would be also no problem using whole subject
categories as reference; here, the download is not possible for a normal
user (as I stated already in my previous email).
With kind regards,
More information about the SIGMETRICS