AW: [SIGMETRICS] SIGMETRICS Digest - 21 Oct 2008 to 22 Oct 2008 (#2008-217)

[Bornmann Lutz]

RE: RE: Study on peer review of grant applications by Dr. Lutz Bornmann

Please find our paper attached.

You are right. In the interpretation of our results it cannot be ruled out that the applicants who received funding from EMBO may have published more subsequent to application because they received funding and not necessarily because the committee made the right choice about who received funding. The higher productivity of the approved applicants against the rejected applicants may be because the committee made the right choice in deciding who should get funding but also be because they had funding allowing them (better) opportunities for research and subsequent publishing. We know that almost all rejected applicants did their research (proposed in the application) with the money of other funding organizations.

However, we did not only consider number of publications in our bibliometric analyses, but also citations counts. Here we have the same results as with number of publications: For an approved applicant, the expected scientific mean performance is statistically significantly increased by 41% (citations for papers published prior to application) and by 49% (citations for papers published subsequent to application) against a rejected applicant. In view of citations I cannot make a connection between received EMBO funding and impact of papers. In my view it is not possible that the name of a certain funding organization in the acknowledgement of a paper can systematically increase citation counts.

Best,
Lutz

-----Ursprüngliche Nachricht-----
Von: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at listserv.utk.edu] Im Auftrag von David Watkins
Gesendet: Donnerstag, 23. Oktober 2008 11:43
An: SIGMETRICS at listserv.utk.edu
Betreff: Re: [SIGMETRICS] SIGMETRICS Digest - 21 Oct 2008 to 22 Oct 2008 (#2008-217)


RE: Study on peer review of grant applications by Dr. Lutz Bornmann

Surely it should be no surprise that people selected for prestigious
fellowships 'perform' better. The direct resources and status accruing will
assit them in both in actively researching and in acheiving publication.
The psychological impact will also be positive.

Isn't this just a further example of the 'Matthew Effect' rather than a
test of the effectivenes of committee peer review? The surprise is that
some of those selected STILL don't get the flying start one might
anticipate. That might seem to indicate that the process is rewarding some
very weak candidates.

I've only seen the Abstract in the Listserv so far. Presumably these issues
are discussed in the paper itself.

Best wishes

David Watkins
Professor and Chair, Postgraduate Research Centre
Southampton Solent University, UK

david.watkins at solent.ac.uk
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Datei.pdf
Type: application/pdf
Size: 280610 bytes
Desc: Datei.pdf
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20081023/7e907552/attachment.pdf>