Robinson, KA; Goodman, SN. 2011. A Systematic Examination of the Citation of Prior Research in Reports of Randomized, Controlled Trials. ANNALS OF INTERNAL MEDICINE 154 (1): 50-U187

[Eugene Garfield]

Robinson, KA; Goodman, SN. 2011. A Systematic Examination of the Citation of 
Prior Research in Reports of Randomized, Controlled Trials. ANNALS OF 
INTERNAL MEDICINE 154 (1): 50-U187..

Author Full Name(s): Robinson, Karen A.; Goodman, Steven N.
Language: English
Document Type: Article

KeyWords Plus: CLINICAL-PRACTICE GUIDELINE; GENERAL MEDICAL JOURNALS; 
LOW-BACK-PAIN; RELEVANT EVIDENCE; REFERENCE BIAS; REVIEWS; SURGERY; 
PUBLICATION; SECTIONS; ARTICLES

Abstract: Background: A randomized, controlled trial (RCT) should not be 
started or interpreted without accounting for evidence from preceding RCTs 
addressing the same question. Research has suggested that evidence from 
prior trials is often not accounted for in reports of subsequent RCTs.
Objective: To assess the extent to which reports of RCTs cite prior trials 
studying the same interventions.
Design: Meta-analyses published in 2004 that combined 4 or more trials were 
identified; within each meta-analysis, the extent to which each trial report 
cited the trials that preceded it by more than 1 year was assessed.
Measurements: The proportion of prior trials that were cited (prior research 
citation index), the proportion of the total participants from prior trials that 
were in the cited trials (sample size citation index), and the absolute number of 
trials cited were calculated.
Results: 227 meta-analyses were identified, comprising 1523 trials published 
from 1963 to 2004. The median prior research citation index was 0.21 (95% CI, 
0.18 to 0.24), meaning that less than one quarter of relevant reports were 
cited. The median sample size citation index (0.24 [CI, 0.21 to 0.27]) was 
similar, suggesting that larger trials were not selectively cited. Of the 1101 
RCTs that had 5 or more prior trials to cite, 254 (23%) cited no prior RCTs and 
257 (23%) cited only 1. The median number of prior cited trials was 2, which 
did not change as the number of citable trials increased. The mean number of 
preceding trials cited by trials published after 2000 was 2.4, compared with 1.5 
for those published before 2000 (P < 0.001).
Limitation: The investigators could not ascertain why prior trials were not cited, 
and noncited trials may have been taken into account in the trial design and 
proposal stages.
Conclusion: In reports of RCTs published over 4 decades, fewer than 25% of 
preceding trials were cited, comprising fewer than 25% of the participants 
enrolled in all relevant prior trials. A median of 2 trials was cited, regardless of 
the number of prior trials that had been conducted. Research is needed to 
explore the explanations for and consequences of this phenomenon. Potential 
implications include ethically unjustifiable trials, wasted resources, incorrect 
conclusions, and unnecessary risks for trial participants.

Addresses: Johns Hopkins Sch Med, Baltimore, MD USA; Johns Hopkins 
Bloomberg Sch Publ Hlth, Baltimore, MD USA
Reprint Address: Robinson, KA, Johns Hopkins Univ, Div Gen Internal Med, Dept 
Med, 1830 E Monument St,Room 8069, Baltimore, MD 21287 USA.

E-mail Address: krobin at jhmi.edu
ISSN: 0003-4819
URL: http://annals.org/content/154/1/50.full