Open access and impact factor (fwd)

[Garfield, Eugene]

Dear Heather Morrison: A colleague sent me your comments since I am
subscribed to this listserv.

Your questions are interesting and valid, but they have been discussed in
the literature of citation analysis for many years. Author, journal and
institutional impact is regular discussed in the ASIS&T listserv at
SIGMETRICS at listserv.utk.edu




---------- Forwarded message ----------
Date: Wed, 10 Mar 2004 23:24:29 EST
From: Heather Morrison <heatherm at eln.bc.ca>
Reply-To: liblicense-l at lists.yale.edu
To: liblicense-l at lists.yale.edu
Subject: Re: Open access and impact factor

Interesting questions.

Is impact itself, important though it might be, really a good measure of
merit? THERE HAVE BEEN NUMEROUS STUDIES WHICH SHOW A HIGH CORRELATION
BETWEEN PEER JUDGMENTS AND CITATION FREQUENCY. KNOWLEDGEABLE CITATION
ANALYSTS WOULD NOT SUGGEST THAT CITATION IMPACT ALONE BE USED TO MAKE SUCH
JUDGMENTS, ESPECIALLY WHEN LOW IMPACT JOURNALS AND AUTHORS ARE INVOLVED.
HOWEVER, MY EXTENSIVE STUDIES OF NOBEL LAUREATES DEMONSTRATES THAT THEY ARE
ALMOST INVARIABLY SUPER CITATION STARS. THESE CAN BE SEEN AT MY WEB PAGE AT
WWW.EUGENEGARFIELD.ORG ESPECIALLY AT
http://www.garfield.library.upenn.edu/prize/prize.html

For example:  if an area in science is suddenly deemed to have economic
significance, and therefore attracts reseach funding, so that articles in
that area are suddenly cited much more frequently, does this mean their
merit has increased? RENEWED INTEREST IN OLD RESEARCH TOPICS IS NOT UNUSUAL.
WITHOUT FUNDING THERE WOULD BE LITTLE RESEARCH SO IT IS TO BE EXPECTED THAT
MORE NEW PAPERS WILL INCREASE CITATIONS TO OLDER ONES. BUT THIS GENERALLY IS
VERY SELECTIVE.
THIS MIGHT BE CONSIDERED A VARIANT ON THE PHENOMENON OF DELAYED RECOGNITION.
THE LATTER SUBJECT HAS RECENTLY BEEN DISCUSSED BY WOLFGANG GLANZEL AND WILL
BE THE TOPIC OF A FORTHCOMING COMMENTARY IN "THE SCIENTIST."


To put it another way, if an environmental scientist writes a definitive
article about a species of mosquito which is about to become extinct, and
no one reads it because no one cares, does that prove that the article has
no merit?  Or, if the importance of the species is suddenly understood 50
years later and citations begin to appear, has merit increased? I THINK THAT
MOST PEOPLE WOULD AGREE THAT ALMOST ANYTHING PUBLISHED HAS "MERIT" (
ESPECIALLY IN THE AUTHOR'S EYES) BUT IF NO ONE CARES AND NO ONE USES IT WHAT
IS ITS RELEVANCE TO THIS DISCUSSION. SCHOLARSHIP IS A MERITOCRACY BUT IT IS
THE IMPACT ON OTHER SCHOLARS THAT MAKES PUBLISHED WORK SIGNIFICANT.

If an article receives many, many citations as an interesting example of
academic fraud, does this mean it has merit? WHILE EXPERIENCE TELLS US THERE
ARE VERY FEW EXAMPLES OF SUCH HYPOTHETICAL CASES ALL THAT MANY CITATIONS
WOULD MEAN IN SUCH A CASE THAT THERE WAS A LOT OF INTEREST IN THAT TOPIC ON
THE PART OF THE SCIENTIFIC COMMUNITY. SUCH FRAUDS ARE INEVITABLY UNCOVERED
AND HAVE LITTLE BEARING ON THE GENERAL DESIRE TO IDENTIFY THAT SMALL
PERCENTILE OF WORK WHICH HAS HIGH IMPACT.

Do articles that are within-paradigm receive more citations than articles
reflecting the view of an emerging paradigm?  Does this reflect merit?
Could an over-reliance on impact as a measure of merit lead to increasing
conservatism within scholarly endeavours? I DOUBT THAT SUCH A STUDY WOULD BE
POSSIBLE CONSIDERING THE DIFFICULT OF DEFINING PARADIGMS. HOWEVER YEAR BY
YEAR CO-CITATION ANALYSIS HAS PERMITTED THE IDENTIFICATION OF EMERGING
PARADIGMS, THAT IS, RESEARCH FRONTS. IT WOULD BE EXTREMELY DIFFICULT TO
DEMONSTRATE WHETHER USE OF CITATION IMPACT INCREASES OR DECREASES
CONSERVATISM. IN THESE MATTERS "BEAUTY IS IN THE EYE OF THE BEHOLDER"

If scholarly information becomes totally open access, and citations to
scientific research are found to be much less than citations to popular
music, does this prove that popular music has greater academic merit or
importance? ALL IT DEMONSTRATES, IF TRUE, IS THAT POPULAR MUSIC IS OF
GREATER UTILITY OR INTEREST THAN SCIENTIFIC RESEARCH.

Do articles that are written in languages which are read by fewer people
of instrinsically less value or merit than articles in more common
languages? HAS ANYONE EVER MADE SUCH A CLAIM? HOWEVER, THAT HAS NOTHING TO
DO WITH UNIVERSAL IMPACT. IT MAY WELL TURN OUT THAT SOME CHINESE ARTICLES
ARE HEAVILY CITED BY CHINESE SCHOLARS, BUT THAT WILL NOT GET THEM READ BY
ENGLISH READERS UNLESS THEY AE TRANSLATED. OF COURSE, THAT SITUATION MAY
CHANGE IN SOME DECADES IF CHINESE BECOMES THE LINGUA FRANCA OF SCIENCE.

Impact - or usage - are easy means of measuring the value of scholarly
information, but not necessarily the best.  Wouldn't it be better to rely
on more objective means of determining merit?  That is the one of purposes
of peer review, is it not? CITATION ANALYSIS IS OBJECTIVE. PEER REVIEW IS
SUBJECTIVE. IF THERE ARE MORE OBJECTIVE MEANS OF DETERMINING MERIT LET'S
HEAR THEM. THE REASON THAT CITATION IMPACT HAS BECOME MORE POPULAR IS
PRECISELY BECAUSE ONE KNOWS WHAT THE MEASUREMENT MEANS. IT IS A MATTER OF
USING THE DATA IN A KNOWLEDGEABLE WAY AND NOT MINDLESSLY.

If work that is important is not being read, do we bury it, cancel our
subscriptions - or educate readers? WHO IS TO DETERMINE THAT WORK IS
IMPORTANT IS NO ONE IS READING IT? THERE IS A HUGE AMOUNT OF LITERATURE THAT
IS NEVER CITED. ARE WE GOING TO PAY MORE ATTENTION TO IT OR TO THE WORK THAT
IS USED AND CITED.
EUGENE GARFIELD

Thanks for raising some interesting issues, Rick.

cheers,

Heather Morrison



________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs Email
Security System.

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________