BTMH User's guide to understanding medical reports
Summary
What should you believe, and how concerned should you be, when reading media reports of new medical and scientific studies? Learn from our practical tips on points to look for.
Key Points
Declaration of interests: No conflict of interests declared
WHAT CAN I BELIEVE ABOUT MEDICAL REPORTS?
Published evidence is increasingly seen as the driving force behind modern medical practice (evidence-based medicine). Accordingly, Behind the Medical Headlines offers readers some advice on reading papers in medical journals or reports on medical papers in the media. There are no certain ways to assess reliability, particularly for non-specialists, but asking yourself a few questions usually indicates what level of credibility can be given to a particular report. First, a few points on medicine and medical advances, second, some points on the nature of research, and finally, a few points on individual reports. Remember that this applies as much to our reports as to any others!
Medicine and medical advances
-
almost nothing in medicine is certain;
-
experts are human, disagree with one another, make mistakes, and change their minds;
-
medicine usually advances slowly against a background of more or less established knowledge;
-
unimportant information may be published and important information may not; journals tend to overlook studies giving negative results, and commercial considerations may prevent publication;
-
scientific ‘breakthroughs’ changing medical opinion overnight are rare and need independent confirmation;
-
implementing new findings into routine medical practice often takes years.
Origin and publication of research
Where was the work done? The most reliable work tends to be done in well-known universities, medical schools, and scientific institutions.
Where was it published? Front-line journals tend to contain the most reliable articles. Competition to publish in these journals is intense, articles are peer-reviewed (judged) by experts before acceptance, declarations of interests are sought, and editors can be very selective about what they publish. Media reports should quote their sources.
What is a front-line journal? These are the most highly regarded journals in their fields. Papers in such journals tend to be quoted by scientists and this increases their impact. ‘Impact factors’are published by the Institute of Scientific Information (ISI) in the form of Journal Citation Reports (JCRs), and while these are only rough measures of quality, they give some idea of a journal’s standing.
Have the authors declared any conflicts of interests such as funding from interested parties (such as pharmaceutical companies)? Such conflicts do not invalidate a report, but they should make a reader cautious.
Was the report presented at a medical meeting or congress? Unpublished data/findings from sources such as these should be regarded as provisional until they are published.
Assessing individual reports
What kinds of research are there? Quantitative research requires numerical, usually statistical, analysis and has underpinned most of the advances in understanding disease, diagnosis, treatment, and outlook over the last half century. Most research studies selected groups of subjects, but epidemiological research looks for disease associations by studying large populations. Qualitative research is now being used increasingly to explore complex issues such as human behaviour, illness experiences, barriers to effective care, reasons for health choices, and human interaction in treatment.
What kind of report am I looking at? Research papers report new knowledge, and meta-analyses combine results from several previously published reports hoping that firmer new knowledge can be confirmed or uncovered. Reputable guidelines are evidence-based and their recommendations are those of the group writing them. Reviews and consensus reports collate established information and give the collective opinions of their writers based on this information.
What question did the research set out to answer? Information answering this question is most reliable, while information discovered by chance, such as by analysing subgroups within the population studied, is less reliable and requires further testing.
What humans did the study include? Results in people who are well may not apply to the sick and may not apply between the sexes, across races, at all ages, and to pregnant women.
Did a study in humans include reasonable numbers of patients and control groups (who did not receive an intervention or treatment) where appropriate, and were the results analysed statistically? Omission of any of these seriously undermines credibility.
Are results expressed in ways that make sense in ‘real life’? A study of 10,000 people may find that an intervention produces an adverse effect in four people as opposed to two in the control group. This could be reported, correctly but misleadingly, as a 100% increase for the general population!
Does the study report an association or does it prove cause and effect? Associations are common, proof of cause and effect is much rarer, and speculation or extrapolation beyond the results is always unreliable.
Was the work done in animals? Animal results cannot be extrapolated to humans uncritically, and animal results take years to translate into new treatments in humans.
What is missing from most media reports is information about the body of knowledge and experience underlying the report. Providing such background information is the objective of Behind the Medical Headlines.