ART: Ojasoo, Citation bias in medical journals

[Gretchen Whitney]

T. OJASOO, J.C. DORE'. Citation bias in medical journals. Scientometrics,
45(1999)81-94.

E-mail:  t.ojasoo at anaes.fr

Background and aim: The impact factor is a scalar measure calculated from
the Institute of Scientific Information's (ISI) citation counts which
ranks journals according to citation frequency. Despite its
attractiveness, it can be misleading. Instead of reducing citation to a
numerical indicator, we used several complementary multivariate methods to
provide objective pictorial descriptions of  the data in order to discover
hidden correlations.

Methods:  We used the citation data for urology and related journals in
the JCR microfilms (ISI database for 1993) to calculate citation profiles
(i.e. the relative extent to which each journal cites itself and other
journals within a set). We analyzed these profiles by three multivariate
methods (hierarchical classification, minimum spanning tree, and
correspondence factor analysis).

Conclusions: (reprinted by kind permission of Scientometrics)

Citation counts contain hidden information on citation habits that can be
unearthed by appropriate descriptive multivariate methods. Because we are
dealing with probabilities, no single algorithm will lead to a definite
conclusion but complementary methods can provide graphic overviews, from
different angles, that mutually support and refine each others'
conclusions. Admittedly, the data in our demonstration refer to a small
number of journals and to a single year (1993). Nevertheless, we would not
be unduly surprised if many of our conclusions prove to be valid for later
years and, possibly, for disciplines other than urology.

We shall emphasize five points:

(i) The journal with the highest rate of self-citation was J. Urology
which is also the journal with the highest impact factor and the American
Urological Society journal.

(ii) There is a transatlantic rift in citation practice, as reported
elsewhere.16 US publications tend to cite each other. European journals,
even if they may cite US journals frequently, cite their European
neighbours more frequently than US journals do.

(iii) In 1993, the citation profile of The Prostate - a journal which ISI
listed under "Endocrinology and Metabolism" only - was closer to that of
urology journals than was the citation profile of Kidney International,
the nephrology journal with the highest impact factor. Urology journals,
which overall interfaced little with other disciplines, tended to refer to
journals in oncology, general & internal medicine, surgery and nuclear
medicine rather than to journals in nephrology. Only since 1995 does ISI
include The Prostate in the "Urology & Nephrology" section but it still
classifies urology and nephrology journals under a single heading and
provides a joint list of impact factors. Impact factors vary widely with
discipline and the way in which ISI allocates journals to disciplines is
not transparent. In our opinion, for most purposes, impact factors should
not be compared if they do not refer to the same fields. The determination
and comparison of citation profiles could be a means of classifying
journals before listing impact factors.

(iv) Comparing citation profiles by factorial analysis can help pinpoint
the journals that might interface best between basic and clinical
research.

(v) The use of correspondence factor analysis to study time-series can
highlight the non-linear nature of citation practice and show that
citations, as in the case of urology journals, tend to refer to particular
epochs. Research does not advance on an even keel and this is reflected in
how frequently publications within a certain time-span are cited.
Are these conclusions relevant to the future or will easier and faster
access to reference data banks and abstracting services dramatically
change citation profiles in years to come? Expensive direct online queries
from Europe to, for instance, Medline have been replaced by cheaper
CD-Roms and now by free access on the Internet (PubMed). Will this bridge
the transantlantic rift, in both directions? Probably not, if specialist
journals from minority groups, however original the research they publish
and wherever their home-base (US, Europe or elsewhere), have difficulty in
getting indexed in the largest and most influential databases (Medline,
SCI, etc...) and if research, after a period of internationalisa-tion,
becomes increasingly balkanized.

Multivariate analyses of publication profiles  (% output per discipline)
of the 48 most prolific nations in 18 scientific disciplines, derived from
ISI data, have indicated that, from 1981 to 1992, there was a clear trend
for nearly all nations to conform to an average world "anglo-saxon"
publication profile defined not only by the US but also by England, in
fact by the G7 nations (except Japan).15,17,18 In clinical medicine, will
citation bias to US journals not continue to increase if all tools are
"made in USA"? Is there not a case for trying to find ways of fostering
original research that springs from the diversity of national cultural
traditions? How about more single author exploratory papers on finding new
hypotheses rather than testing them19; cross-speciality journals; and
developing new search tools?


References

16. J. Joyce, S. Rabe-Hesketh, S. Wessely, Reviewing the reviews: the
example of the chronic fatigue syndrome, Journal of the American Medical
Association, 280 (1998) 264-266.
17. J.C. DorŽ, T. Ojasoo, Y. Okubo, T. Durand, G. Dudognon, J.F. Miquel,
Correspondence factor analysis of the publication patterns of 48 countries
over the period 1981-1992. Journal of the American Association for
Information Science, 47 (1996) 588-602.
18. T. Ojasoo, The disciplines of national wealth. Nature (Scientific
correspon-dence) , 391(1998) 628.
19. T. Ojasoo, The scientific author. Would more single author scientific
papers generate new paradigms?
http://www.tribunes.com/tribune/art98/ojas4.htm, April 1998.



<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
Gretchen Whitney, PhD                                     tel 423.974.7919
School of Information Sciences                            fax 423.974.4967
University of Tennessee, Knoxville TN 37996 USA           gwhitney at utk.edu
http://web.utk.edu/~gwhitney/
jESSE:http://web.utk.edu/~gwhitney/jesse.html
SIGMETRICS:http://web.utk.edu/~gwhitney/sigmetrics.html
<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>