How to Read Medical Papers

During 1997, the British Medical Journal published a series of 10 articles that serve to introduce the non-expert researcher to the varieties of medical articles and to the assessment of their values. By compiling the information and the summary points directly from their site, what follows here enables you to easily access each article as you choose.

The Medline Database
Getting your Bearings (Deciding What the Paper Is About)
Assessing the Methodological Quality of Published Papers
Statistics for the Non-statistician I: Different Types of Data Need Different Tests
Statistics for the Non-statistician II: "Significant" Relations and Their Pitfalls
Papers that Report Drug Trials
Papers that Report Diagnostic or Screening Tests
Papers that Tell You What Things Cost (Economic Analyses)
Papers that Summarize Other Papers (Systematic Reviews and Meta-Analyses)
Papers that Go Beyond Numbers (Qualitative Research)

The Medline Database

Trish Greenhalgh, author
British Medical Journal 315 (7101): 180-183, 1997, July 19.

Compiled by the National Library of Medicine, the Medline database currently indexes millions of articles. This article notes common problem solving situations and search strategies.

Summary points:
Not all medical articles are indexed on Medline, and many that are have been misclassified.
Searching by textword can supplement a search by MeSH headings.
To increase the sensitivity of a search, use the "Explode" command and avoid using subheadings.
Scan titles on screen rather than relying on the software to find the most valid or relevant ones.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7101/180

Getting Your Bearings (Deciding What the Paper Is About)

Trish Greenhalgh, author
British Medical Journal 315 (7102): 243-246, 1997, July 26.

Summary points:
Many papers published in medical journals have potentially serious methodological flaws.
When deciding whether a paper is valid and relevant to your practice, first establish what specific clinical question it addressed.
Questions to do with drug treatment or other medical interventions should be addressed by double-blind, randomized controlled trials.
Questions about prognosis require longitudinal cohort studies, and those about causation require either cohort or case-control studies.
Case reports, though methodologically weak, can be produced rapidly and have a place in alerting practitioners to adverse drug reactions.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7102/243

Assessing the Methodological Quality of Published Papers

Trish Greenhalgh, author
British Medical Journal 315 (7103): 305-308; 1997, August 2.

Before changing your practice in light of a published research paper, you should decide whether the methods used were valid. This article considers five essential questions that should form the basis of your decision.

Summary points:
The first essential question to ask about the methods section of a published paper: Was the study original?
The second: Who is the study about?
The third: Was the design of the study sensible?
The fourth: Was systematic bias avoided or minimized?
The fifth, and last: Was the study large enough, and continued long enough, to make the results credible?

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7103/305

Statistics for the Non-Statistician I: Different Types of Data Need Different Tests

Trish Greenhalgh, author
British Medical Journal 315 (7104): 364-366, 1997, August 9.

As medicine leans increasingly on mathematics, no clinician can afford to leave the statistical aspects of a paper to the "experts." This article, and the next in the series, gives a checklist of preliminary questions to help you appraise the statistical validity of a paper.

Summary Points:
In assessing the choice of statistical tests in a paper, first consider whether groups were analyzed for their comparability at baseline.
Does the test chosen reflect the type of data analyzed (parametric or non-parametric; paired or unpaired?)
Has a two-tailed test been performed whenever the effect of an intervention could conceivably be a negative one?
If obscure tests have been used, do the authors justify their choices and provide references?

Correction

British Medical Journal 315 (7109): 675, 1997, September 13.

An author's error appeared in this article. In Table 1, the chi square test is listed as a parametric test. In fact, both the chi square test and Fisher's exact test are non-parametric.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7104/364

Correction URL: http://bmj.bmjjournals.com/cgi/content/full/315/7109/675

Statistics for the Non-Statistician II: "Significant" Relationships and Their Pitfalls

Trish Greenhalgh, author
British Medical Journal 315 (7105): 422-425; 1997, August 16.

This article continues the checklist of questions that will help you to appraise the statistical validity of a paper.

Summary Points:
An association between two variables is likely to be causal if it is strong, consistent, specific, plausible, follows a logical time sequence, and shows a dose-response gradient.
A "P" value of 0.05 means that this result would have arisen by chance on less than one occasion in 20.
The confidence interval around a result in a clinical trial indicates the limits within which the "real" difference between the treatments is likely to lie, and hence the strength of the inference that can be drawn from the result.
A statistically significant result may not be clinically significant.
The results of intervention trials should be expressed in terms of the likely benefit an individual could expect (for example, the absolute risk reduction).

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7105/422

Papers That Report Drug Trials

Trish Greenhalgh, author
British Medical Journal 315 (7106): 480-483; 1997, August 23.

Before accepting the results presented in drug trials, be sure to research the methodology used. Often, the results of uncontrolled trials are expressed in terms of before and after differences in a particular outcome measure. The recent correspondence in the Lancet and BMJ on placebo effects should remind you why uncontrolled before and after studies are the stuff of teenage magazines, not hard science.

Summary Points:
Pharmaceutical "reps" are now much more informative than they used to be, but they may have ignorance of basic epidemiology and clinical trial design.
The value of a drug should be expressed in terms of safety, tolerability, efficacy, and price.
The efficacy of a drug should ideally be measured in terms of clinical end points that are relevant to patients; of surrogate end points are used, they should be valid.
Promotional literature of low scientific validity (such as uncontrolled before and after trials) should not be allowed to influence practice.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7106/480

Papers that Report Diagnostic or Screening Tests

Trish Greenhalgh, author
British Medical Journal 315 (7107): 540-543; 1997, August 30.

Five features are applied to diagnostic (or screening) tests when compared with a "true" diagnosis or gold standard. They are: sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of this jury's performance. A sixth feature, the likelihood ratio, is introduced at the end of the article.

Summary Points:
New tests should be validated by comparison against an established gold standard in an appropriate spectrum of subjects.
Diagnostic tests are seldom 100% accurate (false positives and false negatives will occur).
A test is valid if it detects most people with the target disorder (high sensitivity) and excludes most people without the disorder (high specificity), and if a positive test usually indicates that the disorder is present (high positive predictive value).
The best measure of the usefulness of a test is probably the likelihood ratio-how much more likely a positive test is to be found in someone with, as opposed to without, the disorder.

Correction

British Medical Journal 315 (7113): 942; 1997, October 11.

An error in terminology occurred in this paper. The value described as the negative likelihood ratio and expressed in the formula (1-sensitivity)/specificity is, in reality, not the negative likelihood ratio, but a value which is described by the question, "How much more likely is a negative result to be found in a person with, as opposed to without, the condition?" The negative likelihood ratio is described by the question, "How much more likely is a negative result to be found in a person without, as opposed to with, the condition?" and is expressed by the formula specificity/ (1-sensitivity). In the example given, a negative urine test for glucose does indeed reduce the window cleaner's baseline chances of diabetes to 0.78 of the pretest likelihood, but the negative likelihood ration of the test is the reciprocal of this value-that is, 1.28.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7107/540

Correction URL: http://bmj.bmjjournals.com/cgi/content/full/315/7113/942

Papers That Tell You What Things Cost (Economic Analyses)

Trish Greenhalgh, author
British Medical Journal 315 (7108): 596-599; 1997, September 6.

An economic analysis can be defined as an analysis that uses analytical techniques to define choices in resource allocation. This article is loosely based on a short booklet by Professor Michael Drummond and two of the forerunners to the "User's Guides to the Medical Literature" series.

Summary Points:
An economic analysis should be based on a primary study or meta-analysis that is scientifically valid, reliable, and relevant.
When deciding whether an economic analysis has been done correctly, you should not simply check the arithmetic, but in addition to that, you should consider whether all direct, indirect, and intangible costs and benefits have been included.
In the allocation of limited resources, the comparison of different health states is unavoidable, but instruments for measuring health related quality of life are not as objective as they seem.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7108/596

Papers That Summarize Other Papers (Systematic Reviews and Meta-Analyses)

Trish Greenhalgh, author
British Medical Journal 315 (7109): 672-675; 1997, September 13.

A systematic review is an overview of primary studies which contains an explicit statement of objectives, materials, and methods, and has been conducted according to explicit and reproducible methodology. This contrasts with the methodology of a journalistic review, which is an overview of primary studies that have not been identified or analyzed in a systematic (standardized and objective) way.

Meta-analyses offer synthesized, numerical data and present relevant information on the inclusion criteria, sample size, baseline patient characteristics, withdrawal rate, and results of primary and secondary end points of all the studies included.

Summary Points:
A systematic review is an overview of primary studies that used explicit and reproducible methods.
A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.
Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the review were valid and reliable.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7109/672

Papers That Go Beyond Numbers (Qualitative Research)

Trish Greenhalgh and Rod Taylor, co-authors
British Medical Journal 315 (7110): 740-743; 1997, September 20.

By observing the complexities of human nature, qualitative research provides a holistic view in which the results interpret, or make sense of, phenomena in terms of the meanings people bring to them.

Summary Points:
Qualitative methods aim to make sense of, or interpret, phenomena in terms of the meanings people bring to them.
Qualitative research may define preliminary questions that can then be addressed in quantitative studies.
A good qualitative study will address a clinical problem through a clearly formulated question and will use more than one research method (triangulation).
Analysis of qualitative data can and should be done using explicit, systematic, and reproducible methods.

Article URL: http://bmj.bmjjournals.com/cgi/content/full/315/7110/740