Roundtree, AK; et al Poor reporting of search strategy and conflict of interest in over 250 narrative and systematic reviews of two biologic agents in arthritis: A systematic review JOURNAL OF CLINICAL EPIDEMIOLOGY, 62 (2): 128-137 FEB 2009

[Eugene Garfield]

E-mail Address: msalmazor at mdanderson.org 

Author(s): Roundtree, AK (Roundtree, Aimee Kendall); Kallen, MA (Kallen, 
Michael A.); Lopez-Olivo, MA (Lopez-Olivo, Maria A.); Kimmel, B (Kimmel, 
Barbara); Skidmore, B (Skidmore, Becky); Ortiz, Z (Ortiz, Zulma); Cox, V 
(Cox, Vanessa); Suarez-Almazor, ME (Suarez-Almazor, Maria E.) 

Title: Poor reporting of search strategy and conflict of interest in over 
250 narrative and systematic reviews of two biologic agents in arthritis: 
A systematic review 

Source: JOURNAL OF CLINICAL EPIDEMIOLOGY, 62 (2): 128-137 FEB 2009 

Language: English 

Document Type: Review 

Author Keywords: QUOROM instrument; Systematic review quality; Rheumatoid 
arthritis; Conflict of interest; Infliximab Etanercept 

KeyWords Plus: MODIFYING ANTIRHEUMATIC DRUGS; RANDOMIZED CONTROLLED-
TRIALS; ANTITUMOR NECROSIS FACTOR; FACTOR-ALPHA ANTAGONISTS; PAPER-BASED 
JOURNALS; RHEUMATOID-ARTHRITIS; COST-EFFECTIVENESS; COCHRANE REVIEWS; 
BLOCKING-AGENTS; CROHNS-DISEASE 

Abstract: Objective: To evaluate the quality of reviews about etanercept 
(ETN) and infliximab (IFX), two biologic treatments for rheumatoid 
arthritis (RA).
Study Design: A comprehensive, systematic review, including searches of 
MEDLINE, EMBASE, and other electronic databases and hand-searches for 
published and unpublished literature. Two raters independently examined 
each article and identified systematic reviews as those including either a 
description of: (1) sources for identification and data retrieval; or (2) 
search strategy. They applied the quality of reporting of meta-analyses 
(QUOROM) instrument to systematic reviews.
Results: Of 3,620 total citations, 281 were identified as reviews. Of 
these, 26 (9%) qualified as systematic rather than narrative. Overall, few 
reviews described selection of sources, critical appraisal, or 
quantitative summary or synthesis. Systematic reviews most often failed to 
explain validity assessment. Several articles did not disclose authors' 
participation in industry-funded clinical trials. Most reviews published 
in high impact factor and rheumatology journals did not meet many quality 
standards. Significant associations existed between review type (narrative 
vs. systematic) and reported funding (P = 0.05), conflicts of interest (P 
= 0.005), and country of publication (P < 0.0001).
Conclusion: More than 90% of the published reviews were narrative and did 
not report methods and conflicts of interest in sufficient detail, raising 
concerns about selection and reporting bias. (C) 2008 Elsevier Inc. All 
rights reserved. 

Addresses: [Suarez-Almazor, Maria E.] Univ Texas MD Anderson Canc Ctr, 
Unit 1465, Houston, TX 77030 USA; [Roundtree, Aimee Kendall] Univ Houston 
Downtown, Houston, TX USA; [Kimmel, Barbara] Baylor Coll Med, Houston Ctr 
Qual Care & Utilizat Studies, Houston, TX 77030 USA; [Skidmore, Becky] Soc 
Obstetricians & Gynaecologists Canada, Ottawa, ON, Canada; [Ortiz, Zulma] 
Acad Nacl Med Buenos Aires, Inst Invest Epidemiol, Buenos Aires, DF, 
Argentina 
Reprint Address: Suarez-Almazor, ME, Univ Texas MD Anderson Canc Ctr, Unit 
1465, 1515 Holcombe Blvd, Houston, TX 77030 USA. 

E-mail Address: msalmazor at mdanderson.org 

Funding Acknowledgement:  
Funding Agency Grant Number 
Canadian Agency for Drugs and Technologies in Health (CADTH)    
National Institutes of Arthritis, Musculoskeletal and Disease Disorders  5 
K24 AR053593  
Agency for Healthcare Research and Quality (AHRQ)  5 U18 HS016093  
 
A major portion of this work was funded by the Canadian Agency for Drugs 
and Technologies in Health (CADTH). In addition, MSA has a K24 career 
award from the National Institutes of Arthritis, Musculoskeletal and 
Disease Disorders [5 K24 AR053593]. She is also the director of the 
Houston Center for Education and Research on Therapeutics (CERTs), which 
receives core support from the Agency for Healthcare Research and Quality 
(AHRQ) [5 U18 HS016093].
 
Cited Reference Count: 73 

Times Cited: 0 

Publisher: PERGAMON-ELSEVIER SCIENCE LTD 

Publisher Address: THE BOULEVARD, LANGFORD LANE, KIDLINGTON, OXFORD OX5 
1GB, ENGLAND 

ISSN: 0895-4356 

DOI: 10.1016/j.jclinepi.2008.08.003 

29-char Source Abbrev.: J CLIN EPIDEMIOL 

ISO Source Abbrev.: J. Clin. Epidemiol. 

Source Item Page Count: 10 

Subject Category: Public, Environmental & Occupational Health 

ISI Document Delivery No.: 396ED 

ANN INTERN MED 142 : 1019 2005 

*J WIL SONS LTD
COCHR HDB SYST REV I : 2006 

ALDERSON P
Academic recognition of critical appraisal and systematic reviews in 
British postgraduate medical education 
MEDICAL EDUCATION 37 : 386 2003 

ANGELL M
Is academic medicine for sale? 
NEW ENGLAND JOURNAL OF MEDICINE 342 : 1516 2000 

BANSBACK NJ
An overview of economic evaluations for drugs used in rheumatoid 
arthritis - Focus on tumour necrosis factor-alpha antagonists 
DRUGS 65 : 473 2005 

BEGG C
Improving the quality of reporting of randomized controlled trials - The 
CONSORT statement 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 276 : 637 1996 

BENT S
AJIC CURR METHODOL C 34 : 246 2004 

BERKENKOTTER C
DECISIONS AND REVISIONS - THE PLANNING STRATEGIES OF A PUBLISHING WRITER 
COLLEGE COMPOSITION AND COMMUNICATION 34 : 156 1983 

BERO L
Factors associated with findings of published trials of drug-drug 
comparisons: Why some statins appear more efficacious than others 
PLOS MEDICINE 4 : 1001 ARTN e184 2007 

BIONDIZOCCAI GGL
Compliance with QUOROM and quality of reporting of overlapping meta-
analyses on the role of acetylcysteine in the prevention of contrast 
associated nephropathy: case study 
BRITISH MEDICAL JOURNAL 332 : 202 DOI 10.1136/bmj.38693.516782.7C 2006 

BLUMENAUER B
COCHRANE DB SYST REV : 4525 2003 

BLUMENAUER B
COCHRANE DB SYST REV : UNSP CD003785 2002 

BLUMENAUER B
Quality of life in patients with rheumatoid arthritis - Which drugs might 
make a difference? 
PHARMACOECONOMICS 21 : 927 2003 

BRESLOW RA
Quality of reviews in epidemiology 
AMERICAN JOURNAL OF PUBLIC HEALTH 88 : 475 1998 

CHALMERS I
Academia's failure to support systematic reviews 
LANCET 365 : 469 2005 

CHALMERS TC
METAANALYSIS OF CLINICAL-TRIALS AS A SCIENTIFIC DISCIPLINE .2. REPLICATE 
VARIABILITY AND COMPARISON OF STUDIES THAT AGREE AND DISAGREE 
STATISTICS IN MEDICINE 6 : 733 1987 

CHO K
Commenting on writing - Typology and perceived helpfulness of comments 
from novice peer reviewers and subject matter experts 
WRITTEN COMMUNICATION 23 : 260 DOI 10.1177/0741088306289261 2006 

COOK DJ
Systematic reviews: Synthesis of best evidence for clinical decisions 
ANNALS OF INTERNAL MEDICINE 126 : 376 1997 

COOK DJ
METHODOLOGIC GUIDELINES FOR SYSTEMATIC REVIEWS OF RANDOMIZED CONTROL 
TRIALS IN HEALTH-CARE FROM THE POTSDAM CONSULTATION ON METAANALYSIS 
JOURNAL OF CLINICAL EPIDEMIOLOGY 48 : 167 1995 

COOPER RJ
Conflict of interest disclosure policies and practices in peer-reviewed 
biomedical journals 
JOURNAL OF GENERAL INTERNAL MEDICINE 21 : 1248 DOI 10.1111/j.1525-
1497.2006.0598.x 2006 

CULY CR
Etanercept - An updated review of its use in rheumatoid arthritis, 
psoriatic arthritis and juvenile rheumatoid arthritis 
DRUGS 62 : 2493 2002 

DEVRIES C
STEER 1 : 1 2001 

FLETCHER RH
EFFECTIVENESS J PEER : 62 2003 

FONG KY
Immunotherapy in autoimmune diseases 
ANNALS ACADEMY OF MEDICINE SINGAPORE 31 : 702 2002 

GABRIEL SE
A clinical and economic review of disease-modifying antirheumatic drugs 
PHARMACOECONOMICS 19 : 715 2001 

GARDAM MA
Anti-tumour necrosis factor agents and tuberculosis risk: mechanisms of 
action and clinical management 
LANCET INFECTIOUS DISEASES 3 : 148 2003 

GARDNER MJ
AN EXPLORATORY-STUDY OF STATISTICAL ASSESSMENT OF PAPERS PUBLISHED IN THE 
BRITISH-MEDICAL-JOURNAL 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 263 : 1355 1990 

GARFIELD E
CURR CONTENTS 20 : 3 1994 

GOODMAN SN
MANUSCRIPT QUALITY BEFORE AND AFTER PEER-REVIEW AND EDITING AT ANNALS OF 
INTERNAL-MEDICINE 
ANNALS OF INTERNAL MEDICINE 121 : 11 1994 

HOCHBERG MC
Comparison of the efficacy of the tumour necrosis factor alpha blocking 
agents adalimumab, etanercept, and infliximab when added to methotrexate 
in patients with active rheumatoid arthritis 
ANNALS OF THE RHEUMATIC DISEASES 62 : 13 2003 

              
HOMIK JE
An economic approach to health care 
BEST PRACTICE & RESEARCH IN CLINICAL RHEUMATOLOGY 18 : 203 DOI 
10.1016/j.berh.2004.02.001 2004 

HUWILERMUNTENER K
Quality of reporting of randomized trials as a measure of methodologic 
quality 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 287 : 2801 2002 

HYRICH KL
Anti-tumour necrosis factor a therapy in rheumatoid arthritis: an update 
on safety 
ANNALS OF THE RHEUMATIC DISEASES 63 : 1538 DOI 10.1136/ard.2004.024737 
2004 

JADAD AR
SEARCHING THE LITERATURE - BE SYSTEMATIC IN YOUR SEARCHING 
BRITISH MEDICAL JOURNAL 307 : 66 1993 

JADAD AR
Methodology and reports of systematic reviews and meta-analyses - A 
comparison of COCHRANE reviews with articles published in paper-based 
journals 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 280 : 278 1998 

JARVIS B
Etanercept - A review of its use in rheumatoid arthritis 
DRUGS 57 : 945 1999 

JEFFERSON T
Effects of editorial peer review - A systematic review 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 287 : 2784 2002 

JEFFERSON T
Measuring the quality of editorial peer review 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 287 : 2786 2002 

JONES G
The effect of treatment on radiological progression in rheumatoid 
arthritis: a systematic review of randomized placebo-controlled trials 
RHEUMATOLOGY 42 : 6 DOI 10.1093/rheumatology/keg036 2003 

KASSIRER J
TAKE MED COMPLICITY : 2004 

KEATING GM
Infliximab - An updated review of its use in Crohn's disease and 
rheumatoid arthritis 
BIODRUGS 16 : 111 2002 

KEATING GM
Management of rheumatoid arthritis - Defining the role of etanercept 
DISEASE MANAGEMENT & HEALTH OUTCOMES 10 : 17 2002 

KHANNA D
Safety of tumour necrosis factor-alpha antagonists 
DRUG SAFETY 27 : 307 2004 

LAUPACIS A
Systematic reviews: Time to address clinical and policy relevance as well 
as methodological rigor 
ANNALS OF INTERNAL MEDICINE 147 : 273 2007 

LEPORE SJ
Psychological interventions for distress in cancer patients: A review of 
reviews 
ANNALS OF BEHAVIORAL MEDICINE 32 : 85 2006
 
LEXCHIN J
Pharmaceutical industry sponsorship and research outcome and quality: 
systematic review 
BRITISH MEDICAL JOURNAL 326 : 1167 2003 

LOTT H
TECHNICAL COMMUNICAI 5 : 169 1996 

LYSENGWILLIAMS KA
Infliximab - A pharmacoeconomic review of its use in rheumatoid arthritis 
PHARMACOECONOMICS 22 : 107 2004 


LYSENGWILLIAMSON KA
Etanercept - A pharmacoeconomic review of its use in rheumatoid arthritis 
PHARMACOECONOMICS 22 : 1071 2004 

MARKHAM A
Infliximab - A review of its use in the management of rheumatoid arthritis 
DRUGS 59 : 1341 2000 

MCALISTER FA
The medical review article revisited: Has the science improved? 
ANNALS OF INTERNAL MEDICINE 131 : 947 1999 

MEADE MO
Selecting and appraising studies for a systematic review 
ANNALS OF INTERNAL MEDICINE 127 : 531 1997 

MERKESDAL S
Cost-effectiveness of TNF-alpha-blocking agents in the treatment of 
rheumatoid arthritis 
EXPERT OPINION ON PHARMACOTHERAPY 5 : 1881 2004 

MOHER D
Improving the quality of reports of meta-analyses of randomised controlled 
trials: the QUOROM statement 
LANCET 354 : 1896 1999 

MOHER D
Does quality of reports of randomised trials affect estimates of 
intervention efficacy reported in meta-analyses? 
LANCET 352 : 609 1998 

MULLEN PD
The promise and pitfalls of systematic reviews 
ANNUAL REVIEW OF PUBLIC HEALTH 27 : 81 DOI 
10.1146/annurev.publhealth.27.021405.102239 2006 

NURMOHAMED MT
Efficacy, tolerability and cost effectiveness of disease-modifying 
antirheumatic drugs and biologic agents in rheumatoid arthritis 
DRUGS 65 : 661 2005 

ONRUST SV
Infliximab - A review of its use in Crohn's disease and rheumatoid 
arthritis 
BIODRUGS 10 : 397 1998 

OSIRI M
Number needed to treat (NNT): implication in rheumatology clinical 
practice 
ANNALS OF THE RHEUMATIC DISEASES 62 : 316 2003 

PETITTI DB
METAANALYSIS DECISIO : 1999 

RUBIOTERRES C
J MED EC 4 : 19 2001 

RUSSELL A
Current and emerging therapies for rheumatoid arthritis, with a focus on 
infliximab: Clinical impact on joint damage and cost of care in Canada 
CLINICAL THERAPEUTICS 23 : 1824 2001 

RYMER J
ADV WRITING RES WRIT 2 : 1988 

SAUERLAND S
Role of systematic reviews and meta-analysis in evidence-based medicine 
WORLD JOURNAL OF SURGERY 29 : 582 DOI 10.1007/s00268-005-7917-7 2005 

SAUNDERS P
The effects of pre-exam instruction on students' performance on an 
effective writing exam 
WRITTEN COMMUNICATION 20 : 195 DOI 10.1177/0741088303255354 2003 

SELZER J
THE COMPOSING PROCESSES OF AN ENGINEER 
COLLEGE COMPOSITION AND COMMUNICATION 34 : 178 1983 

SHEA B
BMC MED RES METHODOL 6 : 27 2006 

SHEA B
A comparison of the quality of Cochrane reviews and systematic reviews 
published in paper-based journals 
EVALUATION & THE HEALTH PROFESSIONS 25 : 116 2002 

SHEA B
Scope for improvement in the quality of reporting of systematic reviews. 
>From the Cochrane Musculoskeletal Group 
JOURNAL OF RHEUMATOLOGY 33 : 9 2006 

SHEA B
SYSTEMATIC REV HLTH : 122 2001 

SUAREZALMAZOR ME
85 CAN AG DRUGS TECH 2007 

SUAREZALMAZOR ME
INFLIXIMAB ETANERCEP : 2007 

WILSON AE
Journal impact factors are inflated 
BIOSCIENCE 57 : 550 DOI 10.1641/B570702 2007