A Study Showing Gingko Biloba Improved Cognition After Stroke Is an Example of Lord's Paradox

A randomized trial, appearing in the journal Stroke and Vascular Neurology, found that Gingko Biloba – the ubiquitous supplement –improved cognitive function among patients with acute ischemic stroke. Or… did it?

Gingko biloba improved cognitive and neurological functions of acute ischaemic stroke: a randomised controlled trial

The trial brings up an interesting statistical quirk, but one that can have big implications for studies like this.

Researchers in China enrolled 348 patients who had an acute ischemic stroke within the past 7 days. They were randomized 1:1 to the Ginkgo group or usual care.

This is problem number one. No placebo control. Patients knew they were getting Gingko – an ancient herb that might help improve their brain function. That knowledge alone might be enough to move the needle a bit on cognitive tests.

How much was the needle moved?

The primary outcome here was based on the Montreal Cognitive Assessment, or "MOCA".

This is a validated test for cognitive impairment, with higher scores being better – much like the mini mental status exam we know and love.

Take a look at the MOCA scores over time in the two arms of the trial.

Baseline scores are similar, follow-up scores are similar... so Gingko doesn't work?

At the beginning of the trial, the MOCA scores in both arms of the study were about the same – 19 or so. By 180 days later, scores in both groups had gone up a bit – a bit more in the ginkgo group, but not enough to achieve statistical significance.

But I said earlier that Ginkgo "improved outcomes". What gives?

Well, the primary outcome was not just MOCA score, it was change in MOCA score from baseline to 180 days.

What you see here is that change, and indeed, the Gingko group improves more over time than the control group.

But people who got Gingko improved from their baseline score more than people who got usual care. What gives?

 But how is that possible? How can the average MOCA score be the same in both groups, but the change in MOCA score be significantly different? Well this is a statistical phenomenon known as Lord’s paradox and it can happen when one simply examines change scores without taking into baseline scores.

This picture illustrates Lord's paradox.

Demonstration of Lord's Paradox. The two groups basically end up in the same spot, but the Ginkgo group, on any given individual level, improves more.

You can see how, overall, the scores end up in pretty much the same place, but on the individual level, every person in the treatment group does better than every person in the control group.

So what we're seeing is a rather weak effect. A strong ginkgo effect would be visible regardless of how you analyze the data. And the lack of an adequate placebo control is probably enough to account for the small observed benefit.  So for now the data on this particular herb is not fully-flowered.