For the video version, click here.

The Affordable Care Act, among many other things, created the Hospital Readmission Reduction Program, which penalized Medicare reimbursement to hospitals with higher 30-day readmission rates for three common diseases: heart failure, acute myocardial infarction, and pneumonia. The motivation for the program was pretty clear: improve the quality of care by tying reimbursement to good care practices.

Of course, any policy like this may have unintended consequences. A hospital might try to avoid readmitting people that really need to be hospitalized – sending them home from the ER, for example. Or a hospital might hold on to patients longer in the first place, prolonging length of stay to be sure that the patients are really 100% ready for discharge, which would drive up costs and potentially increase the risk of nosocomial disasters.

Most concerningly, individuals who die during the first hospitalization can't be readmitted. You don’t want to inadvertently incentivize practices that trade deaths for readmissions.

This week, a study appearing in the Journal of the American Medical Association gives us the best look yet into the real effects the HRRP has had on patient outcomes.

Researchers, largely based at Yale, examined individuals on Medicare admitted to US hospitals from 2008 to 2014.This amounted to roughly 3 million heart failure admissions, 1.3 million acute MIs, and 2.5 million cases of pneumonia. They then examined the link, on a hospital-by-hospital basis, between changes in readmission rates and changes in 30-day mortality.

In epidemiology, our version of “let’s go to the video tape” is “let’s go to the scatter plot”.  So here it is:

I’m showing you the data for heart failure here, but the results were pretty similar for acute MI and pneumonia. Basically, each dot represents a hospital, and you're seeing the relationship between a hospital's change in readmission rate, and change in 30-day mortality rate. The statistics suggest that hospitals that reduced readmission had a slight reduction in 30-day mortality too, but the signal was pretty small – visible only to statistical software and not the human eye.

But have all of the hypothetical concerns been addressed in this study design? In the primary analysis, part of the risk-adjustment formula used to measure how sick patients were included length of stay. I was worried that if a hospital enacted a program to stretch out length of stay – avoiding discharge for fear of readmission – adjusting for length of stay would bias the results.

I asked lead author Kumar Dharmarajan about that issue.

He noted that they did not examine the relationships between length of stay, readmission rate, and mortality, but that they did examine inpatient mortality in a secondary analysis and again found no increased risk among hospitals with better readmission rates. So even if hospitals are keeping patients longer, it doesn’t seem like this is killing them.

The critical thing to take from this paper is that examining off-target effects of policy decisions are critical. In this case, we have no evidence of unintended consequences from the HRRP. But this is but one policy of many, and in a health system as complicated as ours, prediction of the effects of policy will never be a match for measurement of those effects.