Alma Swan: The OA citation advantage: Studies and results to date

Philip Davis pmd8 at CORNELL.EDU
Fri Mar 12 15:13:59 EST 2010

First, I did not state in my critique of the Swan report 
<> that meta-analysis was Alma's idea, but that this 
was your suggestion (as posted to sigmetrics and other listservs).

Secondly, you keep on trying to turn things back to critiquing my own 
work, as if *"the best defense is a good offense."*  You've posted 5 
rapid responses to the BMJ 2008 paper and another rapid response to the 
BMJ editorial.  I've responded to your concerns and have better things 
to do than engaged in an endless discussion with you when there is 
absolutely no hope of changing your mind.  You can continue to plaster 
the Internet with your critiques and astonishment that I haven't 
responded if this makes you feel better.  I have students to teach and a 
dissertation to write.

--Phil Davis


Stevan Harnad wrote:
> Phil,
> Thanks for the helpful feedback.
> I'm afraid you're mistaken about meta-analysis. It can be a perfectly
> appropriate statistical technique for analyzing a large number of
> studies, with positive and negative outcomes, varying in
> methodological rigor, sample size and effect size. It is a way of
> estimating whether or not there is a significant underlying effect.
> I think you may be inadvertently mixing up the criteria for
> eligibility for meta-analysis with the criteria for a clinical drug
> trial (for which there rightly tends to be an insistence on randomized
> control trials in biomedical research).
> Now I would again like to take the opportunity of receiving this
> helpful feedback from you to remind you about some feedback I have
> given you repeatedly on your own 2008 study --
> the randomized control trial that you suggest has been the only
> methodologically sound test of the OA Advantage so far:
> You forgot to do a self-selection control condition. That would be
> rather like doing a randomized control trial on a drug -- to show that
> the nonrandom control trials that have reported a positive benefit for
> that drug were really just self-selection artifacts -- but neglecting
> to include a replication of the self-selection artifact in your own
> sample, as a control.
> For, you see, if your own sample was too small and/or too brief (e.g.,
> you didn't administer the drug for as long an interval, or to as many
> patients, as the nonrandom studies reporting the positive effects had
> done), then your own null effect with a randomized trial would be just
> that: a null effect, not a demonstration that randomizing eliminates
> the nonrandomized drug effect. (This is the kind of methodological
> weakness, for example, that multiple studies can be weighted for, in a
> meta-analysis.)
> I am responding to your public feedback only here, on the SIGMETRICS
> list, rather than also on your SSP Blog, where you likewise publicly
> posted this same feedback (along with other, rather shriller remarks)
> because I am assuming that you will again decline
> to post my response on your blog, as you did the previous time that
> you publicly posted your feedback on my work both there
> and here -- refusing my response on your blog on
> the grounds that it had already been publicly posted elsewhere
> (namely, here!)...
> -- Stevan Harnad
> PS The idea of doing a meta-analysis came from me, not from Dr. Swan.

More information about the SIGMETRICS mailing list