A new study either confirms magical energy lines in our body exist, or had trouble maintaining adequate blinding. You be the judge.

This week, I need to tell you about a study with results that are simply unbelievable.  And I don’t mean that like these results are amazing, I mean I don’t believe them. But I hope that in discussing them I’ll share some insight on how to interpret all medical studies.

But let’s start with the basics. This was a randomized trial, appearing in JAMA Internal Medicine, of acupuncture for the treatment of chronic stable angina in China.

Around 400 patients with stable angina were recruited. At baseline, they were having an average of around 13 anginal attacks per month.

The participants were randomized into four groups, and here is where it gets interesting. First, there was a real acupuncture group, where two needles were placed on specific points -  here and here – that are supposed to lie on invisible energy lines – or meridians - that are beneficial for the heart according to traditional Chinese medicine.

They called this the disease-affected meridian group. Electric stimulus was applied to the needles as well. A second group got two electric needles applied to different acupuncture points here and here – nonaffected-meridians that are not supposed to be beneficial to the heart. 

A third group got non-electric needles applied to “sham” points – not associated with an energy meridian. And a fourth group got waitlisted – they didn’t get any acupuncture at all.

Anginal attacks decreased in all the groups, but as you can see here they decreased most in the disease-affected meridian group, then the nonaffected meridian group, then the sham group, then the waitlist group.

The explanation offered is that these significant differences demonstrate that, in fact, acupuncture works. It’s not just the placebo effect – there is something special about sticking needles in these specific points that reduces angina more than sticking needles anywhere else. If true, this is a groundbreaking finding – an earth-shattering finding really – that upends everything we think we understand about physiology. 

And that – I can’t believe. I do not believe there are hidden energy fields in our body that can be manipulated with needles. I don’t believe that because if there were they could be directly detected by any variety of modern electromagnetic sensors.

So – how did these results come about?  There are a couple of possibilities.

First, I may be wrong. My belief that undetectable energy paths do not flow through the body, founded on a mechanistic understanding of the universe, may be parochial, small-minded, pompous and limited. I’ve been wrong before and I’ll be wrong again. But there are other possibilities and we need to consider those before we embrace a hypothesis that is inconsistent with observation.

The simplest explanation for these findings is that patients were aware of what group they were in. The study states that participants were blinded to the intervention, but it is quite possible that they could figure it out.  Acupuncture is much more popular in China than the US, and participants may have known that they were getting needles in “heart” associated locations as opposed to non-heart associated locations.

And of course, the acupuncturist knew.

It is possible that, whether consciously or unconsciously, that information was conveyed to the participant. One easy way to test this is to ask the participants, after the trial, to guess what group they were in. As far as I can tell, this wasn’t done.

Another possibility is that there was systematic error in measurement or recording of the data.

The point is that this study, and all studies, need to be interpreted not just by the p-value that pops out at the end, but by the likelihood of the underlying hypothesis. I am encouraged that the recent CMS proposal to fund acupuncture for patients requires enrollment in a randomized trial. I would encourage any such studies to robustly assess the quality of blinding.

If this trial had shown that needles in the disease-affected meridian and non-affected meridians both helped equally, better than non-electric sham stimulation, I’d have less trouble believing it.  But ironically the significant differences between the disease-affected meridian group and the non-affected meridian group in this trial actually undermines all the results for me.

That’s right. This study is too good to be true.

This commentary first appeared on medscape.com.