Whether Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research

[Philip Davis]

Yassine,
Thank you for your response.  I find your odds ratio methodology 
unnecessarily complex and unintuitive but now understand your 
explanation, thank you.

Changing how you report your citation ratios, from the ratio of log 
citations TO the log of citation ratios is a very substantial change to 
your paper and I am surprised that you point out this reporting error at 
this point.  While it normalizes the distribution of the ratios, it is 
not without problems, such as:

1. Small citation differences have very large leverage in your 
calculations.  Example, A=2 and B=1, log (A/B)=0.3

2. Similarly, any ratio with zero in the denominator must be thrown out 
of your dataset.  The paper does not inform the reader on how much data 
was ignored in your ratio analysis and we have no information on the 
potential bias this may have on your results.

Have you attempted to analyze your citation data as continuous variables 
rather than ratios or categories?

--Phil Davis



On Fri, Jan 8, 2010 at 10:06 AM, Philip Davis <[log in to unmask] <http://listserv.utk.edu/cgi-bin/wa?LOGON=A2%3Dind1001%26L%3Dsigmetrics%26T%3D0%26F%3D%26S%3D%26P%3D14079>> wrote:

> it is not clear whether your Odds Ratios are interpreted
> correctly.  Based on Figure 4, OA article are MORE LIKELY to receive zero
> citations than 1-5 citations (or conversely, LESS LIKELY to receive 1-5
> citations than zero citations).
>
> You write: "For example, we can say for the first model that for a one unit
> increase in OA, the odds of receiving 1-5 citations (versus zero citations)
> increased by a factor of 0.957. Figure 4.. (p.9)
>
> Similarly in Figure 4 (if I understand the axes correctly), CERN article are
> more than twice as likely to be in the 20+ citation category than in the 1-5
> citation category, a fact that may distort further interpretation of your
> data as it may be that institutional effects may explain your Mandated OA
> effect.  See comments by Patrick Gaule and Ludo Waltman on the review
> http://j.mp/8LK57u
>
> Philip M. Davis
> PhD Student
> Department of Communication
> Cornell University, Ithaca, NY 14853

Yassine Gargouri replied:

Thank you for your comments.

As noted on page 9 of our draft, in the first model, for a one-unit
increase in OA, the odds of receiving 1-5 citations (versus zero
citations) increased by a factor of 0.957. The dependent variables
are:

Cit_a_0&1-5 = 1 (and not 0 as Davis seems interpret) if the citation
count (minus self-citations) is between 1 and 5

and

Cit_a_0&1-5 = 0 if the citation count (minus self-citations) = 0.

As noted in the paper, we re-analyzed the results with and without
CERN, and the result pattern were the same. If the referees request
it, we will include both analyses.

Also, the formula on page 6 should read:

OM/OS = 1/n * S log(OM/OS)

There was an inadvertent error in how we described (not how we
computed) this formula in the text (and we are grateful for this open
feedback which allowed us to detect and correct it!).

There is an advantage in favor of OM when the log of the ratio is
greater then 0, and in favor of OS otherwise.

The log transformation was used to normalize the data and attenuate
the effect of articles with relatively high citation counts, compared
to the whole sample. For example, to compare mandated OA (OM) with
self-selected OA (OS), we computed the log of the ratio OM/OS for each
journal and then we computed the arithmetic mean of all the logs of
those ratios for each journal.





-- 
Philip M. Davis
PhD Student
Department of Communication
301 Kennedy Hall
Cornell University, Ithaca, NY 14853
email: pmd8 at cornell.edu
phone: 607 255-2124
https://confluence.cornell.edu/display/~pmd8/resume
http://scholarlykitchen.sspnet.org/author/pmd8/