Tobin MJ "Impact factor and the journal" AMERICAN JOURNAL OF RESPIRATORY AND CRITICAL CARE MEDICINE 168 (6): 621-622 SEP 15 2003

[Eugene Garfield]

M. Tobin : MTOBIN2 at lumc.edu



TITLE         Impact factor and the Journal
AUTHOR    Tobin MJ
JOURNAL   AMERICAN JOURNAL OF RESPIRATORY AND CRITICAL CARE MEDICINE    168 (6): 621-622 SEP 15 2003


Document type: Editorial Material    Language: English
Cited References: 36                      Times Cited: 1



KeyWords Plus:
CITATION ANALYSIS, SCIENCE, SOCIETY, QUALITY, POLICY

Publisher:
AMER THORACIC SOC, 1740 BROADWAY, NEW YORK, NY 10019-4374 USA

IDS Number:
721PK

ISSN:
1073-449X

THE AUTHOR HAS SHARED THE FULL TEXT OF THE ARTICLE WITH MEMBERS OF THE LIST....

Impact Factor and the Journal
Martin J. Tobin, Editor


The Journal's impact factor rose by 0.611 units in 2002 to reach 6.567 (Figures 1 and 2) . This is the  largest ever one-year increase for AJRCCM.
What does it mean?

These  appear as a graph and bar chart in the actual paper.
DATA:


Year                     Impact Factor

1992                       5.2
1993                       4.7
1994                       4.9
1995                       5.1
1996                       5.0
1997                       4.7
1998                       5.3
1999                       5.5
2000                       5.4
2001                       6.0
2992                       6.6

Figure 1. The impact factor for AJRCCM each year between 1992 and 2002. The impact factor has  increased by 26% since 1998.



Figure 2. The impact factors for journals listed in the categories respiratory system and critical care medicine by the Institute for Scientific
Information for 2002. AJRCMB = American Journal of Respiratory Cell and Molecular Biology; AJP:Lung = American Journal of Physiology: Lung;
ERJ = European Respiratory Journal; JAP = Journal of Applied Physiology; and ICM = Intensive Care Medicine.

Journal                    Impact Factor
AJRCCM                     6.57
AJRCMB                     4.17
Thorax                        4.07
AJP Lung                    3.90
Crit Care Med              3.36
Chest                         2.97
ERJ                            2.93
JAP                            2.72
ICM                            2.04

Figure 2. The impact factors for  journals listed in the categories respiratory system and critical care medicine by the Institute for Scientific
Information for 2002. AJRCMB =  American Journal of Respiratory Cell and  Molecular Biology; AJP:Lung = American Journal of Physiology:
Lung; ERJ = European Respiratory  Journal; JAP = Journal of Applied  Physiology; and ICM = Intensive Care Medicine.


The Institute for Scientific Information has published impact factors of biomedical journals since 1974 (1, 2). Despite serious limitations, this
citation metric has become the most widely used index for ranking the overall quality of biomedical journals. Why are editors so attentive to
impact factor (3-12)? They see science as a collective and cooperative enterprise: an edifice of knowledge built by contributions from successive
generations of researchers (13). Each new article is based on the preceding research that the author cites, and a published article is of little
significance unless other scientists cite it and build on it. (Between 22% and 46% of medical research articles attract not even one citation [14,
15].) When authors cite an article, it usually means they found it helpful when pursuing their own research. Editors recognize that researchers are
 in effect casting votes on the importance of a journal every time they cite it (16).

Listing references at the end of an article is a trademark of scholarship. Within a year of its launch in 1665, authors were placing references in the
 margins of the Philosophical Transactions (17). Footnotes first appeared in 1683 and remained the preferred method of referencing until the
numbers grew too large. Endnotes first appeared in 1876 and are now the standard (17). Along the way, the number of references per article
increased from less than 5 before 1700 to over 40 in 1950.

Investigators cite the work of previous investigators for several reasons: motivating new research, according credit for ideas, providing greater
detail on methodology, highlighting errors in previous research, and persuading readers of the legitimacy of a new claim (18). In a survey of 26
authors on the motivation behind more than 900 citations, the significant reasons were to accord positive credit and to demonstrate that the
author was up to date in a research discipline (19). But the dominant reason by far was persuasiveness: an author wanting to establish legitimacy
 for a new research claim. How well a reader is persuaded, however, will depend on the reputation of the scientist being cited and peer-review
standards of the journal in which the research was published. Because it is notoriously difficult to measure scientific quality, governments and
funding agencies-especially outside the United States-rely heavily on impact factor. In Germany and Italy, agencies decide departmental funding
by plugging impact factors into formulae (20). The Spanish parliament has sanctioned this approach (21, 22), and it is claimed to have led to
improved scientific rigor and productivity (21, 22). An equivalent system is used in Quebec (23). In Finland, $7,000 is added to departmental
funding for a unit increase in impact factor of the journal in which each article is published (20). We can rail against the foolishness of such a
reward system, but we also must see that it carries major implications for the future of research in our field. The impact factors of pulmonary and
critical care journals (Figure 2) lie considerably below those for leading journals of other internal medicine subspecialties: Journal of the National
Cancer Institute (14.500), Gastroenterology (13.440), Circulation (10.255), Blood (9.631), Diabetes (8.256), and Arthritis and Rheumatism (7.379),
to list a few. Simple arithmetic predicts that funding for pulmonary and critical care research may fall considerably below that of other clinical
disciplines in European countries.

When confronted by criticism, advocates of a rewards system based on impact factor counter that it is concrete as opposed to being anecdotal
(20, 23-25). Promotion committees may not use the term impact factor explicitly, but they are using it implicitly whenever they base advancement
on number of papers published and journals in which they appeared, rather than a reading of each paper. Even less defensible is the practice of
basing academic promotion on the number of dollars a researcher received in funding, without checking whether the money yielded publications of
any importance (26). On a list of journals within a particular research field, advocates of bibliometric indexes contend that a journal's impact factor
will correlate well with the quality of research it publishes (20). Recent research supports the contention. When 264 physicians (about evenly
divided between researchers and practitioners) were asked to rate the quality of nine general medical journals, impact factor explained 82% of the
variation in ratings (r = 0.91) (16). The high correlation may reflect not only the reputation of a journal but may truly reflect scientific quality. In
another study of 243 original research articles from 30 journals, an objective measure of scientific quality was more closely related to the journal's
citation rate and impact factor than to any other variable (27).

Impact factor is measured as a ratio. The numerator of the impact factor for AJRCCM in 2002 was the number of citations in all journals in 2002 to
articles that had been published in AJRCCM in 2000 and 2001 (8,708). The denominator was the number of articles published in AJRCCM in 2000
and 2001 (757 + 569 = 1,326). Although pleased that the Journal's impact factor increased by 0.513 in 2001 and by 0.611 in 2002 (Figure 1), both
numbers were brought down by the unusually large number of articles published in 2000. In that year, we published 26% more articles than in
1999 so as to shorten our publication lag-then headed for 10 months. (Our publication lag is now less than 3 months.) In contrast to the impact
factors for 2001 and 2002, the denominator next year will not include the articles of 2000. The formula for impact factor captures three main
qualities of a journal (28). One, the relevance of the journal to researchers in an active field of inquiry. Two, the ability of editors to discriminate
between submissions of the highest quality and the next tier. Three, the speed that the journal publishes articles after acceptance. Only the first
two relate to scientific quality. Nevertheless, speedy publication of accepted articles is also important for the advance of science. If all pulmonary
and critical care journals were to achieve a publication lag of three months or less, the impact factor of every journal in our disciplines would
increase.

In 2000, AJRCCM experienced an increase in submissions of almost 300-equal to the total annual submissions of many pulmonary journals.
Faced with a simultaneously growing publication lag, the Journal necessarily tightened its standards. This step was greatly aided by the
introduction of online submissions, enabling us to make the selection of reviewers more scientific. Through use of descriptor numbers and an
online database of over 6,000 reviewers, we now maximize the match between manuscript content and reviewer expertise. As I have emphasized
on several occasions (13, 29), peer review is the primary instrument for ensuring quality control at AJRCCM (30). And the expectations and
demands of scientists who serve as reviewers are the major determinants of the standing of AJRCCM among other scientific journals (31).
The limitations of impact factor have been discussed by many (20, 23, 32). It is ironic that science has not produced a more scientific tool for
measuring scientific quality. But deficiency in assessing excellence is not limited to science. "We pretend that so many courses, so many
credits, so many books read add up to an education. The same is true of research. We seem immensely satisfied with the outer husk of the
enterprise-the number of dollars spent, the number of publications. Why do we grasp so desperately at externals? Partly because we are more
superficial than we would like to admit," lamented John Gardner (33) (Secretary of Health, Education, and Welfare in the Johnson administration).
How true. Much more important than impact factor, we were pleased to learn from the recent survey of the American Thoracic Society that
members considered AJRCCM as the first among all services provided by the Society (34-36).

FOOTNOTES
Conflict of Interest Statement: M.J.T. is editor of AJRCCM and is responsible for ensuring its quality. He  receives a fixed stipend from the
American Thoracic Society and is not affected financially by the impact factor of the Journal.
REFERENCES
1. Garfield E. Citation analysis as a tool in journal evaluation. Science 1972;178:471-479.
2. Garfield E. Significant journals of science. Nature 1976;264:609-615.
3. Sieck GC. The "impact factor": what it means to the impact of applied physiology. J Appl Physiol 2000;89:865-866.
4. Knox AJ, Britton J. Journal impact factors for 2000: Thorax flying yet higher. Thorax  2001;56:587.
5. Decramer M. The new European Respiratory Journal: alive and kicking. Eur Respir J 2001;17:325-327.
6. Horgan A. BMJ's impact factor increases by 24%. BMJ 2002;325:8.
7. Marban E. Circulation Research impact factor sets new record. Circ Res 2001;89:101.
8. SoRelle R. Circulation impact factor is highest ever. Circulation 2001;104:1450.
9. Opthof T, Coronel R, Janse MJ. Submissions, editorial process and impact factor 1992-2000: focus on Europe.
              Cardiovasc Res 2000;47:203-206.
10. Harder DR. Impact factors and the competitive nature of journal publishing. Am J Physiol Heart Circ Physiol 2000;279:H457.
11. Anonymous. Individual impact factors for AJP journals in 2000. Physiologist 1999;42:391.
12. Lemasters JJ. Impact factors: a good thing? Gastroenterology 2003;124:286.
13. Tobin MJ. Taxonomy of AJRCCM, a new series, and a medley of metaphors. Am J Respir Crit Care Med 2001;164:1333-1335.
14. Schwartz C. The rise and fall of uncitedness. Coll Res Libr 1997;48:19-29.
15. Hamilton DP. Research papers: who's uncited now? Science 1991;251:25.
16. Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality? J Med Libr Assoc 2003;91:42-46.
17. Allen B, Qin J, Lancaster FW. Persuasive communities: a longitudinal analysis of references in the Philosophical Transactions of the
           Royal Society, 1665-1990. Soc Stud Sci 1994;24:279-310.
18. MacRoberts MH, MacRoberts BR. Problems of citation analysis: a critical review. J Am Soc Inf Sci 1989;40:342-349.
19. Brooks TA. Private acts and public objects: an investigation of citer motivations. J Am Soc Inf Sci 1985;36:223-229.
20. Adam D. The counting house. Nature 2002;415:726-729.
21. Saiz-Salinas JI. Failed professor. Nature 1996;381:186.
22. Jimenez-Contreras E, Delgado Lopez-Cozar E, Ruiz-Perez R, Fernandez VM. Impact-factor rewards affect
              Spanish research. Nature 2002;417:898.
23. Taubes G. Measure for measure in science. Science 1993;260:884-886.
24. Calza L, Garbisa S. Italian professorships. Nature 1995;374:492.
25. Brody S. Impact factor as the best operational measure of medical journals. Lancet 1995;346:1300-1301.
26. Tobin MJ. AJRCCM's policy on duplicate publication: infrequently asked questions. Am J Respir Crit Care Med 2002;166:433-434.
27. Lee KP, Schotland M, Bacchetti P, Bero LA. Association of journal quality indicators with methodological quality of clinical research
          articles. JAMA 2002;287:2805-2808.
28. Tobin MJ. The Journal in 2002. Am J Respir Crit Care Med 2002;166:1153-1156.
29. Tobin MJ. Authors, authors, authors-follow instructions or expect delay. Am J Respir Crit Care Med 2000;162:1193-1194.
30. Hoppin FG Jr. How I review an original scientific article. Am J Respir Crit Care Med 2002;166:1019-1023.
31. Tobin MJ. Rigor of peer review and the standing of a journal. Am J Respir Crit Care Med 2002;166:1013-1014.
32. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314:498-502.
33. Gardner JW. Self-renewal: the individual and the innovative society. New York: WW Norton & Company; 1995. p. 82.
34. Schnapp LM, Matosian M, Weisman I, Welsh CH. A snapshot of pulmonary medicine at the turn of the
              century: the American Thoracic Society membership. Am J Respir Crit Care Med 2003;167:1176-1180.
35. Macklem PT. Is cell and molecular biology divorcing from clinical practice? Am J Respir Crit Care Med 2003;167:1164-1165.
36. Brody JS. What we've got here is a failure to communicate. Am J Respir Crit Care Med 2003;168:415-416.