Systematic Review vs Meta-Analysis: What Each Can Tell You

Search any research compound and you will find something describing itself as a review. Some of those are rigorous. Some are a reading list with opinions attached. The systematic review vs meta-analysis distinction is the first tool for telling them apart — and it is narrower, and more useful, than most people assume.

The two words are not synonyms and they are not a ranking. One describes how the evidence was found and appraised. The other describes one thing you can optionally do with it. This guide sets out what each actually is, how to check whether a paper lives up to its label, and — using reviews of compounds in this catalogue — what the answers look like in practice.

Systematic review vs meta-analysis: the actual difference

Start with the definitions as the bodies that set them write them, because the precision matters.

Cochrane describes a systematic review as one that “attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question”, adding that reviewers “use explicit, systematic methods that are selected with a view aimed at minimizing bias, to produce more reliable findings to inform decision making”. The PRISMA 2020 glossary puts it more briefly: “A review that uses explicit, systematic methods to collate and synthesise findings of studies that address a clearly formulated question.”

Every load-bearing word there is about process: pre-specified criteria, explicit methods, one clearly formulated question.

A meta-analysis is something else entirely. Cochrane: “If the results of the individual studies are combined to produce an overall statistic, this is usually called a meta-analysis”, and in the Handbook, “Meta-analysis is the statistical combination of results from two or more separate studies.” PRISMA’s glossary calls it “A statistical technique used to synthesise results when study effect estimates and their variances are available, yielding a quantitative summary of results.”

So the relationship is one of containment, and it is optional in one direction. Cochrane states plainly that “Not every review in the Cochrane Database of Systematic reviews contains a meta-analysis”, and that pooling “might not be appropriate if the designs of the studies are too different”. The Handbook lists legitimate reasons not to pool: limited evidence, incompletely reported estimates, different effect measures, and bias in the evidence.

It is worth noticing what the definitions of meta-analysis do not say. Neither contains any requirement that the pooled studies were found by a systematic search. The definition is purely statistical. That is why “it’s a meta-analysis” is a weaker signal than it sounds, and why the two terms are not a ladder with meta-analysis on top.

How a systematic review is built

The method is what the name is claiming, so it is the method a reader should check.

A protocol, written first. PROSPERO is an international register of systematic review protocols, developed by the Centre for Reviews and Dissemination at the University of York and funded through the NIHR. Registration is meant to happen before formal screening begins. Its purposes are specific: reducing unplanned duplication, providing transparency “with the aim of minimizing reporting bias” including “an audit trail of major changes to planned methods”, and giving enough detail to “allow comparison of planned methods with the subsequent published review”. That last one is the point. A protocol is what makes it possible to tell whether the analysis was planned or discovered.

A search that is reported in full. PRISMA 2020 is the reporting guideline for all of this: a 27-item checklist in seven sections, with an expanded explanation document, a separate abstract checklist and a redesigned flow diagram. Among the changes from the 2009 version, it now asks for full search strategies for all sources rather than one database, for how many reviewers screened and whether independently, for an explanation of studies that looked eligible but were excluded, and — new in 2020 — for certainty in the body of evidence, competing interests, and availability of data and code.

When results can be pooled — and when they can’t

Combining numbers from different studies only means something if those studies were estimating something comparable. The term for the problem is heterogeneity, and Cochrane defines the statistical kind as “Variability in the intervention effects being evaluated in the different studies”, arising from clinical or methodological diversity.

Three quantities get used to describe it, and they answer different questions.

Cochrane offers rough interpretation bands for I-squared — 0–40% “might not be important”, 30–60% “may represent moderate heterogeneity”, 50–90% “may represent substantial heterogeneity”, 75–100% “considerable heterogeneity” — and the overlap between those bands is deliberate. The Handbook warns in terms that “Thresholds for the interpretation of the I2 statistic can be misleading”, because importance depends on the magnitude and direction of the effects and on the strength of the evidence for heterogeneity. Any paper that treats a single I-squared cut-off as a decision rule has skipped that warning.

The model choice matters too, and it is a question about what you are asking rather than a threshold to clear. A fixed-effect analysis assumes “the true effect of intervention (in both magnitude and direction) is the same value in every study”, so differences between results are chance alone. A random-effects analysis instead assumes the effects “follow (usually) a normal distribution”, and its summary is explicitly an “average intervention effect”. Those are two different quantities, not two settings of the same dial.

Three questions people keep confusing

Reading evidence appraisal gets much easier once these are separated, because each has its own tool and none substitutes for another.

A detail worth carrying: RoB 2 “provides a framework for assessing the risk of bias in a single estimate of an intervention effect”. It rates a result, not a whole trial. The same trial can be at low risk for one outcome and high risk for another.

GRADE’s four levels are defined by how much confidence they license. High means “We are very confident that the true effect lies close to that of the estimate of the effect”. Very low means “We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect”. Three things can rate evidence up rather than down: a large magnitude of effect, a dose-response gradient, and plausible residual confounding that would work against the observed effect. Our guide to clinical trial phases covers where the primary studies feeding all this come from, and how to read a peptide study covers appraising one paper on its own.

Reading a forest plot and a funnel plot

Two graphics carry most of the information in a meta-analysis, and both are routinely misread.

In a forest plot, each study is a square “centred on the point estimate of the result of each study”, with a horizontal line through it showing the confidence interval, “usually, but not always, a 95% confidence interval”. The pooled result sits at the bottom as a diamond. The critical detail is the square: Cochrane states that “The area of the block indicates the weight assigned to that study in the meta-analysis”. A bigger square means more weight, which usually means more precision. It does not mean a bigger effect.

A funnel plot is “a simple scatter plot of intervention effect estimates from individual studies”, and asymmetry in it — a gap where small studies showing nothing should be — is often read as proof of publication bias. Two authoritative sources say not to. Sterne and colleagues: “Funnel plot asymmetry should not be equated with publication bias, because it has a number of other possible causes.” Cochrane: “funnel plot asymmetry should not be considered to be diagnostic for the presence of non-reporting bias.” The other causes include flaws that inflate effects in smaller studies, genuine heterogeneity, an artefactual correlation between certain effect measures and their standard errors, and chance.

There is also a hard floor on when these tests mean anything. Cochrane: “Tests for funnel plot asymmetry should be used only when there are at least 10 studies included in the meta-analysis”, and more may be needed when heterogeneity is substantial. Egger’s test — “a test based on a weighted linear regression of the effect estimates on their standard errors” — is the oldest of them, and Cochrane notes the original version is not recommended for odds ratios or standardised mean differences. Most peptide meta-analyses are nowhere near ten studies, which means publication bias in this literature is usually neither demonstrated nor excluded.

What the peptide reviews actually say

Abstract rules land better against real papers. Every row below was checked at the publisher or at Cochrane, and the point is the shape of the evidence, not whether any compound works.

None of that says anything about what these compounds do. It says what kind of paper is standing behind each claim — which is the question this article is about. Our guides to Cerebrolysin, BPC-157 safety, GHK-Cu and KPV go through the underlying literature compound by compound, and the retatrutide phase 3 results covers the primary trials behind that last row.

Six things commonly got wrong

1. “A meta-analysis is the highest level of evidence.” The authors of the revised evidence pyramid argue systematic reviews should come off the apex entirely and be used “as a lens through which other types of studies should be seen”, because a meta-analysis of strong trials “cannot be equated with a meta-analysis of observational studies at higher risk of bias”. Cochrane puts the same point bluntly: “if these results are biased, then a meta-analysis of the studies will produce a misleading conclusion.”
2. “High I-squared means a lot of heterogeneity.” It means a high proportion of the observed variability is not chance. Tau-squared is the quantity. And Cochrane says the interpretation thresholds themselves “can be misleading”.
3. “It says systematic review on the front.” See the GHK-Cu row above, where the paper contradicts its own title in its Methods. Check for a registered protocol, full search strategies, a named risk-of-bias tool and per-outcome certainty — four things that take a minute to look for.
4. “A Cochrane review means it works.” Cochrane’s own 2023 Cerebrolysin stroke review concluded the opposite, at moderate certainty, and flagged a potential increase in non-fatal serious adverse events.
5. “Most of the studies were positive, so it works.” Cochrane is unusually direct here: “Synthesis methods that involve vote counting based on statistical significance have serious limitations and are unacceptable.” Underpowered studies get counted as showing no benefit, study size is ignored, and as the number of studies grows “the power of conventional vote counting tends to zero”.
6. “The funnel plot was asymmetric, so there’s publication bias.” Asymmetry has several other causes, and the tests need at least ten studies to be used at all.

Historically, this is not a pedantic concern. The 1992 Antman study compared expert review recommendations against cumulative meta-analysis of the trials available at the time; as a later JAMA commentary describes it, the recommendations “had not reflected the best contemporary research evidence”, “both effective and dangerous treatments had been overlooked”, and the episode provided evidence that “traditional, unsystematic, narrative reviews did not serve patients well”.

What’s settled and what isn’t

Judging evidence before you buy reagents

If a supplier, a forum post or a product page cites “a systematic review” or “a meta-analysis” behind a compound, four checks take about a minute and settle most of it.

• Find the paper itself, not the summary of it. A claim repeated through three layers of secondary writing is where the detail goes missing — as in the BPC-157 and GHK-Cu examples above.
• Count the primary human studies, not the records. “64 studies included” and “three human studies that directly tested the compound” can describe the same paper.
• Look for the four markers: a registered protocol, full search strategies, a named risk-of-bias tool, and certainty rated per outcome.
• Read the certainty rating before the effect estimate. A statistically significant pooled result rated very low certainty is a different object from the same number rated high.
• Keep the evidence question separate from the material question. What the literature shows and what is in a particular vial are two different problems; the second one is answered by a certificate, which our COA guide explains how to read.

BPC-157One systematic review, 35 of 36 studies preclinical, no meta-analysisView product GHK-CuTwo papers labelled systematic reviews; one self-describes as narrativeView product KPVNo systematic review located — the primary literature is smallView product

Frequently asked questions