Cohen AM, Hersh WR, Peterson K, Yen PY "Reducing workload in systematic review preparation using automated citation classification " Journal of the American Medical Informatics Association 13 (2): 206-219 MAR-APR 2006

[Eugene Garfield]

A.M. Cohen
E-mail : cohenaa at ohsu.edu

Title: Reducing workload in systematic review preparation using automated
citation classification

Author(s): Cohen AM, Hersh WR, Peterson K, Yen PY

Source: JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION 13 (2):
206-219 MAR-APR 2006

Document Type: Review     Language: English
Cited References: 25      Times Cited: 0

Abstract:
Objective: To determine whether automated classification of document
citations can be useful in reducing the time spent by experts reviewing
journal articles for inclusion in updating systematic reviews of drug
class efficacy for treatment of disease.
Design: A test collection was built using the annotated reference files
from 15 systematic drug class reviews. A voting perceptron-based automated
citation classification system was constructed to classify each article as
containing high-quality, drug class-specific evidence or not. Cross-
validation experiments were performed to evaluate performance.

Measurements: Precision, recall, and F-measure were evaluated at a range
of sample weightings. Work saved over sampling at 95% recall was used as
the measure of value to the review process.

Results: A reduction in the number of articles needing manual review was
found for 11 of the 15 drug review topics studied. For three of the
topics, the reduction was 50% or greater.

Conclusion: Automated document citation classification could be a useful
tool in maintaining systematic reviews of the efficacy of drug therapy.
Further work is needed to refine the classification system and determine
the best manner to integrate the system into the production of systematic
reviews.

KeyWords Plus: EVIDENCE-BASED MEDICINE; COMPARATIVE EFFICACY;
CATEGORIZATION; PERCEPTRON; RETRIEVAL; SAFETY
Addresses: Cohen AM (reprint author), Oregon Hlth & Sci Univ, Sch Med,
Dept Med Informat & Clin Epidemiol, 3181 SW Sam Jackson Pk Rd,Mail Code
BICC, Portland, OR 97239 USA
Oregon Hlth & Sci Univ, Sch Med, Dept Med Informat & Clin Epidemiol,
Portland, OR 97239 USA
E-mail Addresses: cohenaa at ohsu.edu
Publisher: ELSEVIER SCIENCE INC, 360 PARK AVE SOUTH, NEW YORK, NY 10010-
1710 USA
Subject Category: COMPUTER SCIENCE, INFORMATION SYSTEMS; COMPUTER SCIENCE,
INTERDISCIPLINARY APPLICATIONS; INFORMATION SCIENCE & LIBRARY SCIENCE;
MEDICAL INFORMATICS
IDS Number: 023HQ

ISSN: 1067-5027

CITED REFERENCES :
*OR HLTH SCI U
DRUG EFF REV PROJ ME : 2003
 APHINYANAPHONGS Y
Text categorization models for high-quality article retrieval in internal
medicine
JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION 12 : 207 2005
 BLASCHKE C
Evaluation of BioCreAtIvE assessment of task 2
BMC BIOINFORMATICS 6 : Art. No. S16 2005
 CARROLL JB
AM HERITAGE WORD FRE : 1971
 CHOU R
Comparative efficacy and safety of skeletal muscle relaxants for
spasticity and musculoskeletal conditions: a systematic review
JOURNAL OF PAIN AND SYMPTOM MANAGEMENT 28 : 140 2004
 CHOU R
Comparative efficacy and safety of long-acting oral Opioids for chronic
non-cancer pain: A systematic review
JOURNAL OF PAIN AND SYMPTOM MANAGEMENT 26 : 1026 2003
 COHEN AM
A categorization and analysis of the criticisms of Evidence-Based Medicine
INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS 73 : 35 2004
 COHEN AM
P 13 TEXT RETR C TRE : 2004
 COHEN W
P ANN C AM ASS ART I : 335 1999
 DIETTERICH TG
Approximate statistical tests for comparing supervised classification
learning algorithms
NEURAL COMPUTATION 10 : 1895 1998
 DOBROKHOTOV PB
Assisting medical annotation in Swiss-Prot using statistical classifiers
INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS 74 : 317 2005
 FREUND Y
Large margin classification using the perceptron algorithm
MACHINE LEARNING 37 : 277 1999
 HAYNES RB
What kind of evidence is it that Evidence-Based Medicine advocates want
health care providers and consumers to pay attention to?
BMC HEALTH SERVICES RESEARCH 2 : Art. No. 3 2002
 HAYNES RB
DEVELOPING OPTIMAL SEARCH STRATEGIES FOR DETECTING CLINICALLY SOUND
STUDIES IN MEDLINE
JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION 1 : 447 1994
 HERSH W
"A world of knowledge at your fingertips": The promise, reality, and
future directions of on-line information retrieval
ACADEMIC MEDICINE 74 : 240 1999
 HERSH WR
P 13 TEXT RETR C TRE : 2004
 JOACHIMS T
SVM LIGHT SUPPORT VE : 2004
 MULROW C
SYSTEMATIC REV SYNTH : 1998
 NELSON HD
Postmenopausal hormone replacement therapy - Scientific review
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 288 : 872 2002
 PORTER MF
PROGRAM 14 : 127 1980
 ROSENBLATT F
THE PERCEPTRON - A PROBABILISTIC MODEL FOR INFORMATION-STORAGE AND
ORGANIZATION IN THE BRAIN
PSYCHOLOGICAL REVIEW 65 : 386 1958
 SACKETT DL
Evidence based medicine: What it is and what it isn't - It's about
integrating individual clinical expertise and the best external evidence
BRITISH MEDICAL JOURNAL 312 : 71 1996
 SACKETT DL
CLIN EPIDEMIOLOGY BA : 1985
 WILCZYNSKI NL
AMIA ANN S P : 719 2003
 WONG SS
AMIA ANN S P : 728 2003