part 2 - Frederic G. Hoppin, Jr. "How I Review an Original Scientific Article" BY Hoppin FG AM J RESP CRIT CARE 166 (8): 1019-1023 OCT 15 2002

[Eugene Garfield]

____________________________________________________________

Part #2  - Frederic G. Hoppin, Jr "How I Review an Original Scientific
Article "
AM J RESP CRIT CARE 166 (8): 1019-1023 OCT 15 2002



Contd. Part 2
Occasional Essay

How I Review an Original Scientific Article

Frederic G. Hoppin, Jr.
Departments of Medicine, Memorial Hospital of Rhode Island and Brown
University, Pawtucket, Rhode Island

Correspondence: Correspondence and requests for reprints should be addressed
to Frederic Hoppin, M.D., Division of Pulmonary and Critical Care Medicine,
Memorial Hospital of Rhode Island, 111 Brewster Street, Pawtucket, RI 02860.
E-mail: Frederic_Hoppin_Jr at Brown.edu
____________________________________________________________________________

Paper continues here....

THE WRITE-UP

Comments to the Editors
I aim for three concise sections as outlined below, totaling no more than
200 words. My experience as Associate Editor was clear: descriptions were
more useful than a bare recommendation or any checklist of numerical
responses (8).

The summary, in three or four sentences, identifies the topic of the study,
indicates the basic approach, selects the main findings, and paraphrases the
authors' main conclusions. The summarizing exercise is important to me for
distilling my thoughts, and it provides the Associate Editor with the
background for the main criticisms/questions that follow.

I then list several main criticisms/questions in descending order of
importance. These are selected from among the list on the front page. For
each, I summarize its basis, postponing a full explanation to the "Comments
to the Authors." I also indicate what I see as the importance of each item
and what I think the authors may be able to do in response.

Finally, I indicate and characterize my recommendations, e.g., "This is a
novel idea, worth inviting major revision (see COMMENTS TO THE AUTHORS)." I
also indicate my degree of confidence in my recommendation, e.g., "I suspect
that the authors will have difficulty answering Question 1 satisfactorily."

I acknowledge any major help I have received-not to absolve myself of any
responsibility for the recommendation but to give credit where it is due and
to introduce individuals whom the Associate Editor might find helpful as
reviewer in the future.

I once informed the Associate Editor of a potential conflict of interest
when I reviewed a paper that reported the characteristics of a device in
which the author had a personal commercial interest and that might have been
considered to be less than fully objective.

Comments to the Authors
This may be 1,500 words or more in length, although probably the best
reviews I have seen have been shorter, comprising a few sentences that posed
clear, insightful, targeted questions.

Some ground rules: (1) my observations and analysis must be clear to the
authors. The recommendations themselves, however, are private, and should
not preempt the Associate Editor; (2) I downplay praise. If the article is
accepted, the authors will be pleased enough, regardless of any praise. If
the article is rejected and I had praised it, the Associate Editor may have
an uncomfortable interchange with the authors at the next national meeting;
(3) I avoid censure, even of chronic offenders, as it is unnecessary,
belittling, and disrespectful. Each submission is on the order of a
man-year's work by my fellow scientists, and the stakes are high for them.

The first paragraph is a direct copy of my summary at the beginning of
"Comments to the Editor." From this, the authors can learn what I took from
their presentation (possibly a surprise to them), which may help them to
focus and prepare their responses and revision.

If there are aspects of the study that I have not evaluated, I say so
explicitly, e.g., "I have reviewed the applied math carefully for its
assumptions and physiologic implications, but am not equipped to review the
math itself."

This is followed by "Major Comments," numbered and starting with the main
points made in the "Comments to the Editors." Each criticism/question must
be explained to the authors and never left as unsupported, qualitative
statements, such as "inadequate controls"-there must always be a "because
..." I have occasionally indicated alternative approaches, either to explain
a criticism or question or to let the authors pick up a useful suggestion.
This practice carries the risk that the alternatives may reflect my own
scientific style or arbitrary choices rather than an objective scientific
evaluation.

With regard to presentation, I have often heard it said that referees have
limited time and responsibility, that deficiencies of presentation are the
authors' responsibility, that it is presumptuous for authors to submit
articles without first obtaining meticulous, critical reviews from in-house
or out-house colleagues, that submissions should always be gone over
carefully by someone for whom English is the native tongue, and that any
negative consequences of weak and hasty presentation are well deserved. I
sympathize deeply with each of these arguments, but the overriding
consideration is that good science should be published. One can call the
authors on sloppiness by giving them a few egregious examples and
generalizing the complaint. It is more challenging to help with poor
exposition, e.g., it is difficult to explain why I do not understand what I
do not understand! The best path through this particular thicket starts with
telling the authors what I did understand from what they wrote, e.g., "this
appears to say ... ," or with telling them where I got lost, e.g., "at this
point I found myself wondering ..." Once, exasperated with a chronic
offender, who routinely presented novel science very badly, I pointedly
suggested a rigorous, in-house, line-by-line revision by his senior author.
Once, having had to work hard to untangle an argument in an otherwise
worthwhile paper, I wrote, "Is this what you are saying?" and outlined my
version. The authors accepted it fully. And gratefully (or so they said). I
have never come close, however, to doing what Jere Mead once did, namely
completely rewriting a foreign paper, believing that otherwise some
excellent science would have been lost!

For many specific problems with the presentation, it may be useful to refer
the authors to online sites (17).

The final section is "Minor Comments." This is culled from my notations in
the text, (i.e., about redundancies, inappropriate symbols, and the like)
listed in the order they appear in the text and identified by page,
paragraph, and line.

The issue of reviewer anonymity is controversial (11). Most journals presume
that the referee prefers anonymity and that this will help ensure relative
objectivity. Many of us, however, sign some of our reviews if we think the
authors might welcome direct dialog later.

FINAL COMMENT

I asked a colleague recently how his recent submission had fared in review.
"One bad review, one good review," he replied. "The 'bad' one liked it but
was really superficial-I don't think they understood it. The 'good' one
didn't like it much, but the review was just wonderful." By which he meant
it was really insightful and helpful to the quality of his science.

Acknowledgments

Uninhibited in-house reviews of an early draft of this article by Robert
Banzett and James Butler were very helpful and warmly appreciated.

Received in original form April 15, 2002; accepted in final form July 25,
2002

REFERENCES

1. Rennie D. Fourth International Congress on Peer Review in Biomedical
Publication. JAMA 2002;287:2759-2760.
2. Journal of the American Medical Association. 2002;287. Issue 21.
3. Siegelman SS. Assassins and zealots: variations in peer review: special
report. Radiology 1991;178:637-642.[Medline]
4. Lock S. A difficult balance: editorial peer review in medicine, 1st ed.
London: Nuffield Provincial Hospitals Trust; 1985.
5. Kassirer JP, Campion EW. Peer review: crude and understudied, but
indispensable. JAMA 1994;272:96-97.[Medline]
6. Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good
reviewer and a good review for a general medical journal? JAMA
1998;280:231-233.[Medline]
7. Evans AT. The characteristics of peer reviewers who produce good-quality
reviews. J Gen Intern Med 1993;8:422-428.[Medline]
8. Goodman SN, Berlin J, Fletcher SW, Fletcher RH. Manuscript quality before
and after peer review and editing at Annals of Internal Medicine. Ann Intern
Med 1994;121:11-21.[Medline]
9. Day RA. How to write and publish a scientific paper, 4th ed. Phoenix, AZ:
Oryx Press; 1994.
10. Huth E. How to write and publish papers in the medical sciences, 2nd ed.
Baltimore: Williams and Wilkins Co.; 1990.
11. Godlee F. Making reviewers visible: openness, accountability, and
credit. JAMA 2002;287:2762-2765.[Medline]
12. Weber EJ, Katz PP, Waeckerle JF, Callaham ML. Author perception of peer
review: impact of review quality and acceptance on satisfaction. JAMA
2002;287:2790-2793.[Medline]
13. Altman DG. Poor-quality medical research: what can journals do? JAMA
2002;287:2765-2767.[Medline]
14. Altman DG, Goodman SN, Schroter S. How statistical expertise is used in
medical research. JAMA 2002;287:2817-2820.[Medline]
15. Comroe JH Jr, Dripps RD. Scientific basis for the support of biomedical
science. Science 1976;192:105-111.[Medline]
16. Comroe JH Jr. The soul of wit. Am Rev Respir Dis
1975;112:861-866.[Medline]
17. On line at www.atsjournals.manuscriptcentral.com.






When responding, please attach my original message
_______________________________________________________________________
Eugene Garfield, PhD.  email: garfield at codex.cis.upenn.edu
home page: www.eugenegarfield.org
Tel: 215-243-2205 Fax 215-387-1266
President, The Scientist LLC. www.the-scientist.com
Chairman Emeritus, ISI www.isinet.com
Past President, American Society for Information Science and Technology
(ASIS&T)  www.asis.org
_______________________________________________________________________