Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P, Bhandari M "Level of evidence in orthopaedic journals" Journal of Bone and Joint Surgery - American Volume 87A (12): 2632-2638 Dec. 2005

[Eugene Garfield]

E-mail Addresses: william.obremskey at vanderbilt.edu


Title: Level of evidence in orthopaedic journals

Author(s): Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P, Bhandari M

Source: JOURNAL OF BONE AND JOINT SURGERY-AMERICAN VOLUME 87A (12): 2632-
2638 DEC 2005

Document Type: Article
Language: English
Cited References: 22      Times Cited: 0

Abstract:
Background: The American edition of The Journal of Bone and Joint Surgery
(JBJS-A) has included a level-of-evidence rating for each of its clinical
scientific papers published since January 2003. The purpose of this study
was to assess the type and level of evidence found in nine different
orthopaedic journals by applying this level-of-evidence rating system.

Methods: We reviewed all clinical articles published from January through
June 2003 in nine orthopaedic journals. Studies of animals, studies of
cadavera, basic-science articles, review articles, case reports, and expert
opinions were excluded. The remaining 382 clinical articles were randomly
assigned to three experienced reviewers and two inexperienced reviewers,
Who rated them with the JBJS-A grading system. Each reviewer determined
whether the studies were therapeutic, prognostic, diagnostic, or economic,
and each rated the level of evidence as I, II, III, or IV. Reviewers were
blinded to the grades assigned by the other reviewers.

Results: According to the reviewers' ratings, 70.7% of the articles were
therapeutic, 19.9% were prognostic, 8.9% were diagnostic, and 0.5% were
economic. The reviewers graded 11.3% as Level I, 20.7% as Level II, 9.9% as
Level III, and 58.1% as Level IV. The kappa values for the interobserver
agreement between the experienced reviewers and the inexperienced reviewers
were 0.62 for the level of evidence and 0.76 for the study type. The kappa
values for the interobserver agreement between the experienced reviewers
were 0.75 for the level of evidence and 0.85 for the study type. The kappa
values for the agreement between the reviewers' grades and the JBJS-A
grades were 0.84 for the level of evidence and 1.00 for the study type. All
kappa values were significantly different from zero (p < 0.0001 for all).
The percentage of articles that were rated Level I or II increased in
accordance with the 2003 journal impact factors for the individual journals
(p = 0.0061).

Conclusions: Orthopaedic journals with a higher impact factor are more
likely to publish Level-I or II articles. The type and level of information
in orthopaedic journals can be reliably classified, and clinical
investigators should pursue studies with a higher level of evidence
whenever feasible.


Addresses: Obremskey WT (reprint author), Vanderbilt Univ, Dept Orthopaed
Surg, 131 Med Ctr S,2100 Pierce Ave, Nashville, TN 37212 USA
Vanderbilt Univ, Dept Orthopaed Surg, Nashville, TN 37212 USA
Boston Univ, Med Ctr, Dept Orthopaed Surg, Boston, MA 02118 USA
McMaster Univ, Dept Clin Epidemiol & Biostat, Hamilton Hlth Sci, Hamilton,
ON L8L 2X2 Canada

E-mail Addresses: william.obremskey at vanderbilt.edu

Publisher: JOURNAL BONE JOINT SURGERY INC, 20 PICKERING ST, NEEDHAM, MA
02192 USA
Subject Category: ORTHOPEDICS; SURGERY
IDS Number: 991JW

ISSN: 0021-9355


CITED REFERENCES :
BHANDARI M
Interobserver agreement in the application of levels of evidence to
scientific papers in The American Volume of The Journal of Bone and Joint
Surgery
JOURNAL OF BONE AND JOINT SURGERY-AMERICAN VOLUME 86 : 1717 2004

 CURTI M
Impact factor and electronic versions of biomedical scientific journals
HAEMATOLOGICA 86 : 1015 2001

 GALLAGHER EJ
Evidence of methodologic bias in the derivation of the Science Citation
Index impact factor
ANNALS OF EMERGENCY MEDICINE 31 : 83 1998

 GARFIELD E
Journal impact factor: a brief review
CANADIAN MEDICAL ASSOCIATION JOURNAL 161 : 979 1999

 HANSSO S
IMPACT FACTOR AS A MISLEADING TOOL IN EVALUATION OF MEDICAL JOURNALS (VOL
346, PG 906, 1995)
LANCET 346 : 1172 1995

 HANSSON S
IMPACT FACTOR AS A MISLEADING TOOL IN EVALUATION OF MEDICAL JOURNALS
LANCET 346 : 906 1995

 HECHT F
The journal "impact factor": A misnamed, misleading, misused measure
CANCER GENETICS AND CYTOGENETICS 104 : 77 1998

 JACSO P
A deficiency in the algorithm for calculating the impact factor of
scholarly journals: The journal impact factor
CORTEX 37 : 590 2001

 KURMIS AP
Current concepts review - Understanding the limitations of the journal
impact factor
JOURNAL OF BONE AND JOINT SURGERY-AMERICAN VOLUME 85 : 2449 2003

 LANDIS JR
MEASUREMENT OF OBSERVER AGREEMENT FOR CATEGORICAL DATA
BIOMETRICS 33 : 159 1977

 LEHRL S
Evaluating scientific performances by impact factors - the right for equal
chances
STRAHLENTHERAPIE UND ONKOLOGIE 175 : 141 1999

 LINDE A
On the pitfalls of journal ranking by impact factor (R)
EUROPEAN JOURNAL OF ORAL SCIENCES 106 : 525 1998

 LUBOWITZ JH
Understanding evidence-based arthroscopy
ARTHROSCOPY-THE JOURNAL OF ARTHROSCOPIC AND RELATED SURGERY 20 : 1 2004

 MANSKE PR
J HAND SURG-AM 29 : 983 2004

 NOWICKI SA
AAOS B 53 : 45 2005

 REIDER B
Read early and often
AMERICAN JOURNAL OF SPORTS MEDICINE 33 : 21 2005

 ROGERS LF
Impact factor: The numbers game
AMERICAN JOURNAL OF ROENTGENOLOGY 178 : 541 2002

 SACKETT DL
RULES OF EVIDENCE AND CLINICAL RECOMMENDATIONS ON THE USE OF ANTITHROMBOTIC
AGENTS
CHEST 89 : S2 1986

 THOMSEN NO
J ORTHOP SCI 7 : 163 2002

 WALSH EF
Spine: Scientific citation index and its impact factor
SPINE 23 : 1087 1998

 WHITEHOUSE GH
Impact factors: facts and myths
EUROPEAN RADIOLOGY 12 : 715 2002

 WRIGHT JG
Introducing levels of evidence to The Journal
JOURNAL OF BONE AND JOINT SURGERY-AMERICAN VOLUME 85 : 1 2003