A flawed study calls the whole syndrome into question.

Post-Acute Covid-19 Syndrome (PASC), better known as Long COVID, or long-haul COVID has been described since the very beginning of the pandemic. But, as syndromes go, this one is really hard to define. On October 6th, the WHO released a consensus case-definition for long COVID that is woefully vague – it defines a syndrome that

-Occurs in those with a history or probable history of SARS-CoV-2 infection with

-Symptoms that occur or persist three months from infection and last at least two months and

-Can not be explained by an alternative diagnosis

What symptoms might count?  Virtually everything, as you can see here. The list includes brain fog, abdominal pain, palpitations, anxiety, and new allergies among others.

And this week, a new article in JAMA Internal Medicine shows data that, frankly, calls into question the very existence of Long COVID.

I’ll show my cards now – I think this study is fairly problematic, and I’ll walk you through the problems in a minute but first a reminder that no one is saying that symptoms do not occur after COVID. The biological question of interest is two-fold. Does COVID cause these symptoms, and, if so, does COVID uniquely cause these symptoms?

In other words, does long-COVID occur because people survived a really bad illness? After all, studies from the before times show that 2/3rds of people who survive an ICU stay have persistent symptoms. But is COVID uniquely bad – uniquely harmful to a variety of organ systems, out of proportion to severity of illness?

The unique finding in the JAMA-IM study? The presence of long-COVID symptoms was more strongly associated with the belief you had COVID, than whether you actually had COVID. The implication, left unsaid: it’s all in your head. Here’s how it worked.

Researchers in France leveraged an existing, population-based cohort study that had been running since around 2012. From May to December of 2020 – right during the peak of the first wave in France, they asked around 36,000 participants to volunteer to send in a blood sample to detect anti-SARS-CoV-2 antibodies. This seroprevalence data tells us if those individuals had prior infection. From December 2020 to January 2021, they asked those same people to tell them about their persistent symptoms, and, critically, whether they thought that they had contracted COVID.

What this created was four groups of people – the negatives – those without serologic evidence of infection who didn’t think they had been infected – this was the overwhelming majority.

Then there were the positives – people who thought they had been infected and, indeed, had serologic evidence to that effect.

And then the more interesting groups – the asymptomatics – people with serologic evidence of infection who never knew they were infected.

And finally a group that thought they had been infected, but had no serologic evidence to that effect.

Now, if long-COVID is caused by and unique to COVID, you would expect that the prevalence of long-COVID symptoms would be higher in people with serologic evidence of COVID infection than those who merely thought they had COVID infection. But that’s not what was seen here.

Here’s a couple of symptoms to illustrate the difference.

In each case, you were more likely to report the symptom if you thought you had had COVID, regardless of whether the serologic evidence supported it or not.

The researchers put both belief you had COVID and serologic evidence of COVID into a multivariable model, allowing them to compete, as it were, to see which, independent of the other, would associate more strongly with symptoms. Across the board, belief bested serology as a predictor of long covid symptoms. Again, the implication – if thinking you had COVID is a better predictor of long COVID than actually having COVID, is long-COVID even real?

But, as I alluded to earlier, there is a real problem with this analysis that was, I think, inadequately addressed in the paper.  It has to do with the serologic test itself. The authors report that the serologic test they used had a sensitivity of 87% for detecting prior infection and a specificity of 97.5%  - good numbers.  But, at the time this study was conducted, very few people in France had been infected – just 4% by the study accounting.  Let’s do some math – I’m sorry – to figure out how the test characteristics interact with the disease prevalence.

Ok – the study included 26,823 individuals. Let’s assume that 4% had a prior COVID infection – that’s 1,073 people. Given a sensitivity of 87%, only 933 of those people would be detected by the test. That leaves 140 false-negatives. They would live up here, somewhere, in the serologically negative group. The authors, to their credit, point out that the false-negatives represent about 1% of the total serologically negative – not enough to move the needle.

But no mention is made of false positives. With 97.5% specificity, we can estimate that 644 individuals would test positive on serology, despite never having COVID.

They would be found in this row, and you can see that they are a BIG group relative to the size of this row overall. In fact, more than half of the serologic positive group was likely a false-positive in this study.

That’s a real problem if we’re going to argue that serologic positivity is not a good predictor of long COVID – after all, half of our seropositive group never had COVID.  That’s going to dilute your effect somewhat, no?

This wouldn’t be a problem if the prevalence of prior COVID infection was higher – so maybe a follow-up study would shed more light, but for now I am in no way ready to say that Long COVID is all in your head.  Rather, this study may just be telling us that serologic testing for COVID is not as good as a patient’s recollection whether they had COVID or not – many of them had PCR tests at the time proving it in fact – that’s something that you’re likely to remember.

To be honest, I’m a bit frustrated with how we’re handling Long COVID right now. The case definition is bad, we have zero diagnostic tests, and papers like this may be used to argue it isn’t even a real problem. The truth is – long covid definitely exists - I know many patients and friends who weren’t deathly ill from COVID and yet had long, lingering, debilitating symptoms.  But we don’t know how common it is. We need to recognize that vague symptoms lead to vague diagnoses – and without clearer criteria, we risk labeling a bunch of people with “long covid” when that’s not what they have at all.  And that does a disservice to everyone because it makes it that much harder to make progress on this disease… whatever it is.

A version of this commentary first appeared on medscape.com.