Evidence quality

How Many Of These Trials Would Pass A Modern Risk-Of-Bias Check

Every article on this blog links a trial and reports what it found. This one asks a different question about the same trials: how good is the evidence, not just what does it say? A weighted mean difference and a confidence interval are not the end of the story. The size of the trial, who funded it, whether it was blinded, and whether its estimate held up as later trials were added all change how much weight a single number deserves.

By GlucoPril Team · Last updated September 27, 2026 · 11 min read · Evidence quality

What “moderate certainty” actually means

An earlier article on this blog cited Cochrane's 2020 review of the phlebotonic drug class — the venous-support family that most of this label's seven extracts belong to. That review pooled 69 randomised trials and concluded, at moderate certainty, that the class slightly reduces oedema and slightly increases adverse events against placebo.

Moderate certainty is a specific rating on a standard scale, not a vague hedge. Cochrane reviews grade the certainty of their own conclusions using the GRADE framework, which starts every body of randomised-trial evidence at high certainty and downgrades it for specific, named reasons: risk of bias in the individual trials, inconsistency between them, indirectness, imprecision, or publication bias. A downgrade to moderate certainty means the reviewers found real reasons, stated in the review, to trust the pooled estimate a little less than a textbook-perfect trial would deserve — not that the finding is unreliable, but that the confidence interval and the point estimate should be read with those specific caveats attached.

A number that moved: horse chestnut, 2002 versus 2012

The earlier article on horse chestnut covered its best single trial, the 1996 Lancet comparison against compression stockings. The pooled Cochrane estimate for horse chestnut's effect on leg volume is a cleaner illustration of how evidence changes as a literature grows, because the same two reviewers published two versions of it a decade apart, and the number moved.

Same two authors, same outcome measure, ten years and two more trials apart. The effect did not disappear, but its estimated size roughly halved and its confidence interval narrowed as the evidence base doubled.

This is not a story about a flawed review. It is closer to the opposite: a well-run systematic review updating itself as new data arrived, which is exactly the mechanism that is supposed to correct an early, smaller-sample estimate that ran a little high. The lesson for a reader is not that the 2002 number was wrong, but that any single early trial, or any early pooled estimate built on a handful of trials, deserves less confidence than a number backed by twice as many.

Sample size: what eight, twenty-nine and seventy-nine people can and cannot tell you

Several of the trials already cited on this blog and its sibling site are genuinely small by modern standards, and it is worth being specific about what a small trial can responsibly claim.

• The hawthorn-digoxin pharmacokinetic trial enrolled 8 healthy volunteers. That is enough to detect a large, consistent pharmacokinetic shift, and it found none — but it is not enough to rule out a smaller interaction, or one that only appears in an older or sicker population than eight healthy adults.
• The grape seed extract safety trial enrolled 29 healthy adults. That is enough to catch a common, obvious safety problem at the doses tested, and it did not find one — but it is a four-week window, in one country's population, which cannot speak to years of daily use or a more diverse group of takers.
• The hawthorn blood-pressure trial in people with type 2 diabetes enrolled 79 people. That is a reasonably sized trial for a phytotherapy study, large enough to detect the blood-pressure effect it reported, though still modest next to the thousands typically enrolled in a pharmaceutical cardiovascular trial.

None of these are disqualifying. Small trials are how a new area of research starts, and a well-designed small trial can still detect a real, sizeable effect. What a small trial cannot do is rule out a small effect, or a rare harm, with any confidence — which is exactly why the gotu kola hepatotoxicity case reports discussed on this blog's safety article were never going to show up in a 79-person efficacy trial built to look at blood pressure, not liver enzymes.

The blinding problem specific to botanical extracts

A well-designed drug trial blinds both the participant and the person assessing the outcome, so that neither knows who received the active treatment. This is harder to do well with a crude botanical extract than with a manufactured pill, because a plant extract can carry a distinctive taste, smell or colour that a matched placebo has to reproduce convincingly. Several of the older phlebotonic trials pooled in the Cochrane reviews cited on this blog predate modern reporting standards that specifically require authors to describe how blinding was achieved and verified, which is one of the named reasons systematic reviews of this literature, including the 2020 phlebotonics review, downgrade their own certainty rating rather than accepting every trial's blinding claim at face value.

Why “adjusted analysis” is a phrase worth noticing

An earlier article on this blog cited a 2025 randomised trial that gave 1,200 mg of gotu kola a day to people with type 2 diabetes for six months, reporting that the adjusted analysis found no between-group difference in HbA1c, fasting glucose or LDL cholesterol. The word adjusted is doing real work in that sentence. An adjusted analysis statistically corrects for differences between the treatment and placebo groups that existed before the trial even started — a baseline imbalance in age, starting glucose level, or body weight, for instance — rather than simply comparing the two groups' raw endpoint averages. When a trial reports an adjusted result rather than a raw one, it is usually a sign that the groups were not perfectly matched at baseline by chance, which is common even in properly randomised trials, and that the authors did the extra statistical work to account for it rather than let an imbalance masquerade as a treatment effect. Readers who skip past that word miss a small but genuine marker of methodological care.

A four-question checklist for reading any supplement trial

None of this requires a statistics degree. Four questions, asked of any trial cited on any supplement website, catch most of what matters.

1. How many people were actually in it? A trial under 30 people can detect a large effect but not rule out a small one.
2. Was it placebo-controlled and, if possible, was blinding described and checked, or just claimed?
3. Is this the only trial, or has it been replicated? A single trial, however well run, is one data point.
4. Did the reported result come from a raw comparison or an adjusted one, and does the write-up say why?

Where that leaves this label’s trial literature

Applying those four questions across the trials already cited on this blog produces a mixed but honest picture. The Cochrane reviews score well: large pooled samples, a named and transparent grading system, and a certainty rating stated up front rather than implied. The individual pharmacokinetic and safety trials score reasonably on blinding and design but are small by nature, which is a real limitation rather than a disqualifying flaw. The 2025 gotu kola glycaemic trial scores well on methodological transparency, given its use of an adjusted analysis. None of this changes what any of these trials actually found, reported elsewhere on this site. It changes how much weight a single number from a single trial deserves before the next one, or the next Cochrane update, either confirms it or moves it, the way horse chestnut's own leg-volume number moved between 2002 and 2012.

References