Published research is at the heart of 21st century healthcare, but even the best research has its limitations.
By Paul Cerrato, MA, senior research analyst and communications specialist and John Halamka, M.D., Diercks President, Mayo Clinic Platform
In previous columns, we have emphasized the importance of scientific evidence as the foundation of modern medicine. Such evidence consists of a hierarchy that includes randomized controlled trials (RCTs), large observational studies, animal experiments, test tube data, and physicians’ clinical experience. Realizing that RCTs are at the top of the list, the consumer press often highlights new trials when they are published in peer-reviewed journals. Unfortunately, many of the write-ups in the popular press forget to put the latest news in context, often forgetting to mention the many studies that came before, both positive and negative. That’s where systematic reviews and meta-analyses need to come into the picture.
A systematic review is a review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review. It differs from a narrative review, which offers readers a descriptive overview of the medical literature without using this strict, exhaustive approach. A meta-analysis, on the other hand, is a statistical technique that researchers use to evaluate the data collected in a systematic review, combining the results from the studies that the review has collected into a single summary of the evidence. That summary can be expressed as a odds ratio, risk ratio, or standard mean difference and confidence interval.
The first step in developing a systematic review is framing a clear question that you want to answer. It will involve a clinically relevant issue, typically about whether a specific treatment will benefit patients with a specific condition. Once that’s been decided, an exhaustive search of the medical and scientific literature has to be done to locate all the relevant studies on the topic. Among the databases to consult:
- Medline/PubMed, the US National Library of Medicine’s bibliographic database for life sciences and biomedicine
- Embase, a biomedical and pharmacological research database from Elsevier
- Cochrane Library, a collection of databases containing medical and healthcare evidence
- Scopus, also from Elsevier, an abstract and citation database focused on health, education, and related topics
- Web of Science, which provides access to scholarly literature in science, arts, and humanities
- CINAHL (Cumulative Index to Nursing and Allied Health Literature), which provides access to nursing literature and related health fields
The next step is to extract relevant data from all the studies you’ve located. That includes making a record of the most important details of each study, including the sample size, i.e, how many patients were enrolled, and the effect of the intervention that was tested, i.e., did it have an impact on patients? Extraction should also look for possible bias and heterogeneity in the collected studies. Major differences in the way individual studies were designed, the patient population involved, drug dosages, and outcomes measured may make it difficult to combine the results and arrive at a meaningful conclusion about whether a specific treatment is effective. A primer on these issues is available elsewhere.
A meta-analysis evaluates all the collected data and generates a forest plot to help busy clinicians visualize the results. The analysis usually involves two stages. As the Cochrane Library explains it, “In the first stage, a summary statistic is calculated for each study, to describe the observed intervention effect in the same way for every study. For example, the summary statistic may be a risk ratio if the data is dichotomous, or a difference between means if the data is continuous. In the second stage, a summary (combined) intervention effect estimate is calculated as a weighted average of the intervention effects estimated in the individual studies.”
While systematic reviews and meta-analyses have provided invaluable information to help clinicians make everyday decisions, they have their limitations, including the fact that they often miss patients that fall through the cracks. If, for example, a certain drug and lifestyle intervention only benefits one in 1,000 individuals but all the studies performed included 300 or fewer patients, it’s possible the studies would jump to the conclusion that the treatment has no value.
Similarly, numerous studies may indicate that the cut-off point for a diagnostic test is 5.0 mg/dl for instance, one might conclude that a patient with levels below that probably doesn’t have the disease. But that reasoning ignores the possibility that a patient may have subclinical or preclinical disease, a phenomenon that has been documented many times over the years. That includes subclinical hypothyroidism and cardiovascular disease. With that in mind, Mayo Clinic and Thermo Fisher Scientific have joined forces to create PreCure to identify signs of disease before symptoms occur.
Randomized controlled trials, systematic reviews, and meta-analyses have transformed medicine. Combining these tools with multi-omics, proteomic, genomic, and longitudinal clinical data promises to take us to the next level of patient care.
Share this:
