A new study shows how the measure is biased against safety-net institutions.

Today I’m going to tell you a story of good intentions, and intentions gone awry.

The setting? Hospitals across the United States. The antagonist? Sepsis.

Sepsis is the leading cause of death among hospitalized patients – responsible for around 300,000 deaths per year in the US. And with sepsis, time is of the essence. The quicker we can diagnose a patient, the quicker we can initiate treatment, and the more likely they are to survive. Our window to save a life in the setting of severe sepsis is measured in hours, not days.

Recognizing the importance of rapid diagnosis and treatment, multiple states – most notably New York - have instituted mandatory sepsis guidelines and public reporting of outcomes. The Center for Medicare Services has recognized this as well – implementing sepsis care as one of their key quality measures. CMS reports data on hospital compliance with sepsis guidelines as well as outcomes. Here’s the data from my hospital, for example.

This is all good, right? Sunlight is the best disinfectant. OK maybe in this setting I should point out that antibiotics are better than sunlight but let’s move on. There’s a wrinkle.

The key outcome these agencies use to determine how good your hospital is at treating sepsis is inpatient mortality. And it’s one of those outcomes that makes a lot of sense – at least until you look at the data.

I am bringing up this issue because of this study, appearing in JAMA Network Open, that sheds some light on the complicated issue of measuring hospital performance.

Source: Law et al. JAMA Network Open. 2024

Researchers used Medicare data to identify 2.5 million older adults admitted to hospitals with sepsis or septic shock between 2011 and 2019. They then explored these patients’ outcomes based on whether they were in a safety-net hospital or not.

Now, patients admitted to safety net hospitals – even patients with sepsis – tend to be sicker overall than those admitted to other hospitals. More comorbidities, worse pre-hospital care. The authors dutifully adjust for these factors and find what CMS and state reporting has shown multiple times before. Even accounting for the different case-mix, inpatient mortality from sepsis is significantly worse at safety-net hospitals. 28.2% of patients with sepsis admitted to a safety-net hospital will die during that admission compared to 26.4% of those admitted to other hospitals.

A damning indictment of our critical safety-net hospitals, right? A failure of the system.

Except… it’s not. This supposed disparity is not actually due to worse care – it’s due to how non-safety-net hospitals deal with dying patients.

Any doctor who spends a lot of time in an ICU will tell you about a few archetypes of hospitalizations that happen there. There are the 24-hour folks – the people that, within the first day in the ICU either mount a complete recovery, or simply can’t be saved. And then there are the slogs – the patients in that liminal space between life and death that persists for days, even weeks. We call them “meta-stable”, and, with good care, hope to see them slowly recover. But sometimes, they take the other trajectory. The slow, inexorable decline – characterized by the gradual up-titration of pressors, the increased minute ventilation, the third-spacing of fluids. The end seems inevitable but hasn’t come yet. What do you do with these patients?

It turns out, if you’re in a non-safety net hospital, you involve palliative care. You talk to the family. You transition the patient to hospice. A patient who dies in hospice care – even when that hospice care is happening within an acute care hospital – does NOT count in the inpatient mortality metric. In a safety-net hospital, where resources like palliative care and hospice are less available? Those patients experience inpatient mortality – the outcome measure CMS reports on.

And that phenomenon turns out to explain virtually all the disparity in sepsis outcomes between safety-net and non-safety net hospitals. As the authors of the study show, if instead of using inpatient mortality as your outcome measure, you use 30-day mortality – the outcomes are virtually identical.

Here is the breakdown of those 30-day deaths, stratified by type of hospital. You can see how non-safety-net hospitals are more likely to transfer the care of the dying patient.

Now – I’m not implying that the reason these hospitals do more transfers to hospice is to game the system. Far from it – in fact I believe appropriate hospice care is a GOOD outcome. But publicly shaming safety-net hospitals for not having access to hospice care really helps no one.

Why not just use 30-day mortality as the outcome measure? We should, honestly. Critics will point out that it is harder to capture – hospitals are quite good at knowing when someone dies under their care- but it can be harder to track down dates of death after the patient is back in the community. Still, this study shows it is possible – with appropriate integration of national and state databases.

So let’s take a moment to appreciate what our safety-net hospitals are doing. With less resources – with poorer staffing – with less access to ancillary services – they are actually doing just as well at treating sepsis as more affluent centers. Given that, it seems that they deserve a lot more respect than they’ve been given.

A version of this commentary first appeared on Medscape.com.

Methods Man Blog

Counterintuitive: Inpatient Mortality is Not a Good Metric To Judge Hospital Performance

A new study shows how the measure is biased against safety-net institutions.

The Methods Man