It Matters. A Lot.

As of this recording, there have been 4.2 million confirmed cases of coronavirus in the US, and 147,000 deaths, for a case-fatality rate of 3.5%.

If you were to rank the countries of the world with respect to case fatality rate you’d find the US doing ok – about on par with Germany, and Brazil. Way better than Italy, and worse than New Zealand and Australia, for instance. This is not to say we are containing the virus well, just that among those who get it, the chance of survival is about as good here as those other countries.

But there are two big problems with the case-fatality rate metric, problems this paper, appearing in PLOS Medicine, attempts to solve.

The first problem gets talked about a lot. The case fatality rate is defined as the number of deaths from COVID-19 divided by the number of cases of COVID-19.

If you assume we aren’t capturing all the cases of COVID-19, and at this point the evidence is overwhelming that we are missing a large proportion of cases, then the case fatality rate becomes really dependent on the denominator – how many people you can test.

We tend to test those who are symptomatic, for example. And even among those with symptoms, those with WORSE symptoms – the type that might land you in the hospital – are more likely to get tested than those with less severe symptoms.

But the second problem with case fatality rate is more insidious – and it has to do with the lag between cases and deaths.

Let’s conduct a completely unethical experiment in our minds.  Take 100 healthy individuals, isolate them on a desert island and infect them all with SARS-CoV-2 on day 1.

What is the case-fatality rate on day 2 on that island?  It’s probably 0.  No one has even developed symptoms yet, much less died.

But, over time, as people get sick and die, the rate climbs – until it flattens out at some final number once everyone is either over the disease or dead from it.

If you captured everyone with the disease (asymptomatic or not) and waited until the whole thing was over to count how many people died from it, you would have the infection fatality rate.

So – case fatality rate – dependent on testing, restricted to a moment in time. Infection fatality rate – the thing we really want to know to tell how good we are at treating people with COVID-19.

The PLOS paper uses a standard susceptible-exposed-infectious-resolved model of epidemic transmission to try to turn case fatality rates into infection fatality rates. It’s an impressive bit of statistical rejiggering that basically uses the characteristics of the observed infections to model what went undetected.

For example, the case-fatality rate in Lombardy Italy, was a dizzying 17.8%. Looking at that figure alone, compared to the global case fatality rate of around 4% we might think the Lombardy was a disaster zone.

And, of course, it was to some extent – we know there was rationing of ventilators for instance – but it probably wasn’t this bad.  Using estimates from the SEIR model, the researchers pegged the infection fatality rate in Lombardy at just 1.4%. 

If the transformation is accurate – this is a much more useful metric to compare quality of medical care in different places, because countries can’t “game the system” by testing more. It can also show us how an individual country is improving in managing these cases over time. Here is the temporal trend in infection fatality rate from Hubei, China – for example.

You can see that while the case-fatality rate was rising (due to that right-censoring problem – deaths were catching up with the initial explosion of the epidemic), the infection fatality rate was falling – evidence that the docs there were getting a bit better at treating COVID-19.

When you compare countries by infection fatality rate instead of case-fatality rate, there is still variation, but it’s not the 4-5 fold difference in death rates you may have heard about.

The authors also do us a service by translating the infection fatality rate across age groups. Here the data is fairly concerning – showing 10-30% mortality – call it “true mortality” if you want – among those 70 years and older.

Now, this may be driven a bit by one of their statistical assumptions – that the ascertainment of cases is much better among older adults than younger people, considering they are more likely to be symptomatic and / or likely to get tested.  In other words, they sort of assume we’re capturing most of the COVID-19 cases in elderly people already, so the infection fatality rate and the case fatality rate are pretty close.  I’m not sure this assumption is totally valid, but it is what it is.

The authors didn’t look at US data in this study, unfortunately, so I can’t tell you how we stack up against the rest of the world using this better metric.

What I can tell you is that the CDC now provides an “infection fatality rate” parameter in it’s planning scenarios.

They range the IFR from 0.5% in their best-case scenario to 0.8% in the worst-case scenario. Given that the calculations of IFR in this study range from 0.5% to 2.5%, we may need to revise our worst-case estimates upwards.

The truth is we probably won’t know the IFR until this is all over. By then it may not be that helpful. But remember, whatever that true rate is, the total deaths will be lower if we can keep more people from being infected.

This commentary first appeared on medscape.com.