Risk adjustment – the soft underbelly of healthcare system reform

Sept. 21, 2016 / By Norbert Goldfield, MD, Richard Fuller, MS

Last month, an article appeared in Modern Healthcare lauding the achievement of New York’s Bellevue Hospital in reducing readmissions to their facility. The article raised a number of issues relating to a study published in Health Affairs by the architects of the CMS hospital readmission risk adjustment model. The Health Affairs study was intended to counter arguments that identify socioeconomic status (SES) as a driver of hospital readmission penalties. The study reported that:

“The percentage of low socioeconomic status hospitals that changed from having an excess readmission ratio from above one to equal to or below one was four percent for acute myocardial infarction, three percent for heart failure, and three percent for pneumonia when Medicaid status was added to the model.”

These numbers were deemed insignificant by the authors of the study.  This position was in turn contested in commentary by the Association of American Medical Colleges (AAMC)  which highlighted the strong association between SES and readmission performance contained in the study results and the financial impact of the perceived bias upon low SES hospitals. Our purpose in writing this blog is to summarize the challenges that researchers have cited in regard to many of the CMS quality metrics that are tied to payment, and subsequent CMS modifications (or lack therefore).

The Health Affairs study also highlighted the link between enrollee complexity (presence of comorbidities) and low SES. Two years ago, we looked at the correlation between hospitals with readmission penalties and comorbidities, applying risk adjustment, using the CMS model. We found that there was significant correlation between penalties, patient complexity and DSH percentage (a widely used measure of the percentage of low-income patients treated by a hospital). We found similar penalty shifting impacts on high DSH hospitals as those reported in the Health Affairs article (13.8 percent vs. 4 percent AMI; 6.2 percent vs. 3 percent pneumonia; 4.5 percent  vs. 3 percent  heart failure) but responsibility for the bias stemmed from inadequacies in the risk adjustment model.

The Modern Healthcare article highlighted the ability of a hospital to improve care for a challenging population—something that should be evident.  However, to do so requires hospital investments in programs and the community that reduce impediments to accessing care (neighborhood pharmacies and primary care resources were highlighted). At the same time, the article did not challenge the CMS argument that because improvements are possible it is fair to hold all facilities to the same standard without making allowance for variation in treating the fraction of more challenging patients with lower SES . As has been highlighted by others, the fact that a subset of hospitals with a high percentage of low SES patients are observed as performing better on the measure than a subset of hospitals with a low percentage of low SES patients is irrelevant as a defense of the measure’s validity. In fact, the authors of the Health Affairs article state that, “Overall, adjustment for socioeconomic status does not change hospital results in meaningful ways.”  However, this leads to the conclusion that community access to care has little or no effect on readmissions. This flies in the face of the bulk of expert opinion^1-5 and common sense – but it is the only way to reconcile the results without faulting the risk adjustment.

While risk adjustment for the CMS readmission policy is back in the news, it is far from the only risk adjustment model that is being challenged. Jeffrey Silber et al have highlighted a flaw in the risk adjustment of the CMS mortality outcomes measure for a number of years^6,7. The flaw being raised is that the mortality risk adjustment model adjusts the outcomes of low-volume hospital providers to be partly represented by an all-hospital average in an effort to improve statistical inference. While the mortality argument is technical in nature, it can be distilled into a simple statement. If you believe that the volume of procedures that a hospital conducts affects its mortality performance, then the risk adjustment model obscures poor performance of low-volume hospitals. Even though this has been long held to be true^8-12 the CMS has not modified its approach – in fact, it was inserted “as is” into the value- based purchasing composite score.

We have also recently published an analysis of the hospital acquired condition reduction program (HACRP) and came to a similar conclusion—the risk adjustment being used is flawed.  In fact, the risk adjustment for HACRP is a topic we covered in an earlier blog.

Finally, the Medicare Advantage risk adjustment model is still (12 years after implementation) characterized as doing little or nothing to reduce incentives to select more profitable enrollees based upon their comorbid chronic condition and likely requirement for complex care^13,14. Others see improvement from the original position but with the potential for inherent selection incentives to remain. Incentives to select more profitable enrollees will become increasingly important as Medicare Advantage margins are reduced.

It’s clear that there has been a pattern of action, or more accurately inaction, in which obvious deficits in the risk adjustment used to assess performance outcome measures are, at best, explained unsatisfactorily but more usually ignored. There needs to be a shift in the CMS thinking to plan for the evolution of risk adjustment models with the understanding that while methods may be imperfect they can be improved going forward if there is an openness to researcher commentary and industry feedback. Returning to readmissions, we would contend that the CMS should acknowledge that SES impacts outcomes (either directly through poverty and access or indirectly through comorbidities) and a re-examination of the CMS risk adjustment model in total is, therefore, necessary given that there is an observable bias against safety net hospitals.  Refusing to adapt the underlying risk adjustment models in light of such strong industry concern undermines the goals of performance measurement.

Richard Fuller, MS, is an economist with 3M Clinical and Economic Research.

Norbert Goldfield, MD, is medical director for 3M Clinical and Economic Research.

References:

1. Hu J, Gonsahn MD, Nerenz DR. Socioeconomic status and readmissions: evidence from an urban teaching hospital. Health Aff (Millwood). 2014;33(5):778-785. doi:10.1377/hlthaff.2013.0816.
2. Joynt KE, Figueroa JE, Oray J, Jha AK. Opinions on the Hospital Readmission Reduction Program: results of a national survey of hospital leaders. Am J Manag Care. 2016;22(8):e287-94. http://www.ncbi.nlm.nih.gov/pubmed/27556831. Accessed September 19, 2016.
3. Medbery RL, Gillespie TW, Liu Y, et al. Socioeconomic Factors Are Associated With Readmission After Lobectomy for Early Stage Lung Cancer. Ann Thorac Surg. July 2016. doi:10.1016/j.athoracsur.2016.05.060.
4. Oronce CIA, Shao H, Shi L. Disparities in 30-Day Readmissions After Total Hip Arthroplasty. Med Care. 2015;53(11):924-930. doi:10.1097/MLR.0000000000000421.
5. National Quality Forum. Risk Adjustment for Socioeconomic Status or Other Sociodemographic Factors.; 2014. http://www.qualityforum.org/Publications/2014/08/Risk_Adjustment_for_Socioeconomic_Status_or_Other_Sociodemographic_Factors.aspx.
6. Silber JH, Satopää VA, Mukherjee N, et al. Improving Medicare’s Hospital Compare Mortality Model. Health Serv Res. 2016;51 Suppl 2:1229-1247. doi:10.1111/1475-6773.12478.
7. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume-outcome relationship. Health Serv Res. 2010;45(5 Pt 1):1148-1167. doi:10.1111/j.1475-6773.2010.01130.x.
8. Hannan EL, Kilburn H, O’Donnell JF, et al. A longitudinal analysis of the relationship between in-hospital mortality in New York State and the volume of abdominal aortic aneurysm surgeries performed. Health Serv Res. 1992;27(4):517-542. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1069892&tool=pmcentrez&rendertype=abstract. Accessed September 12, 2016.
9. Rathore SS, Epstein AJ, Volpp KGM, Krumholz HM. Hospital coronary artery bypass graft surgery volume and patient mortality, 1998-2000. Ann Surg. 2004;239(1):110-117. doi:10.1097/01.sla.0000103066.22732.b8.
10. Patti MG, Corvera CU, Glasgow RE, Way LW. A hospital’s annual rate of esophagectomy influences the operative mortality rate. J Gastrointest Surg. 2(2):186-192. http://www.ncbi.nlm.nih.gov/pubmed/9834415. Accessed September 12, 2016.
11. Birkmeyer JD, Siewers AE, Finlayson EVA, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002;346(15):1128-1137. doi:10.1056/NEJMsa012337.
12. Begg CB, Cramer LD, Hoskins WJ, Brennan MF. Impact of hospital volume on operative mortality for major cancer surgery. JAMA. 1998;280(20):1747-1751. http://www.ncbi.nlm.nih.gov/pubmed/9842949. Accessed September 12, 2016.
13. Rahman M, Keohane L, Trivedi AN, Mor V. High-Cost Patients Had Substantial Rates Of Leaving Medicare Advantage And Joining Traditional Medicare. Health Aff (Millwood). 2015;34(10):1675-1681. doi:10.1377/hlthaff.2015.0272.
14. Goldberg EM, Trivedi AN, Mor V, Jung H-Y, Rahman M. Favorable Risk Selection in Medicare Advantage: Trends in Mortality and Plan Exits Among Nursing Home Beneficiaries. Med Care Res Rev. August 2016. doi:10.1177/1077558716662565.