OA Growth Monitoring Needs a Google Data-Mining Exemption

[Stevan Harnad]

This is a response to a query* *regarding Eric Archambault's report on OA
Growth<http://www.science-metrix.com/pdf/SM_EC_OA_Availability_2004-2011.pdf>
 by Adam G Dunn<http://news.sciencemag.org/scientific-community/2013/08/free-papers-have-reached-tipping-point-study-claims#comment-1014280880>
 in *Science Insider*: *"I find it difficult to believe that the authors of
the study managed to create a harvester that could identify and verify the
pdfs linked to by Google Scholar when Google Scholar actively blocks IP
addresses when they identify crawling*."

Our own "harvester" attempts to gather the all-important data on OA growth
were blocked by Google.

It is completely understandable and justifiable that Google shields its
increasingly vital global database and search mechanisms from the countless
and incessant worldwide attempts at exploitation by commercial interests,
spammers, and malware that could bring Google to its knees if not
rigorously and relentlessly blocked.

But in the very special (and tiny) case of scientific research articles it
would not only be a great help to the worldwide research community but to
Google (and Google Scholar) itself if Google granted special individual
exemptions for important international studies like Eric Archambault's,
which was commissioned by the European Union to monitor the global growth
rate of open access to research.

Google and Google Scholar would become all the richer as research databases
if data like Eric's (and our own) were not made so excruciatingly difficult
and time-consuming to gather by Google's blanket blockage of automated
data-mining.

(We do not trawl books, so Google's agreements with publishers are not
violated or at issue in any way. We just want to trawl for articles whose
metadata match the the metadata from Web of Science or SCOPUS and have been
made freely accessible on the web; nor do we want their full-texts: just to
check whether they are there!)

Stevan Harnad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20130823/6188157c/attachment.html>