The need to validly compare nursing home quality across state lines

Aug. 24, 2018 / By Richard Fuller, MS

June and July saw a flurry developments highlighting the challenges with the five star nursing home compare ranking system. In June, the Veterans Administration released their star rankings for VA managed facilities, lauding both their new found transparency and what was interpreted as positive performance relative to private sector nursing homes¹. Within a week the same stars reporting mechanism was being used to demonstrate that VA nursing homes in fact had lower quality of care than their private sector counterparts – particularly in areas of clinical care such as the administration of antipsychotic medications². 

In July, the CMS began using audited payroll-based journal (PBJ) data to calculate stars in the nursing domain³. The changes in star ratings of using the newer verified staffing data was picked up on by the New York Times and Kaiser Health News with their joint review of nursing home stars and payroll data. It has been of long standing concern that self-reported staffing levels compiled by nursing homes were neither sufficiently standardized nor independently verified. The importance of this in calculating the nursing home stars can be observed by Medicare revising star ratings for 9 percent of nursing homes as a result of auditing the staffing data⁴.

There are many positives in having a public discussion of the quality of care in nursing homes but, as with the stars for hospital compare, this is only helpful if the measures upon which the discussion is based reflects what a person seeking care would consider to be high quality. There are serious questions over what exactly nursing home compare stars show. For the VA, the assumed overuse of antipsychotic medications was explained by differences in patient mix compared to private sector nursing homes (e.g. a comparatively large prevalence of PTSD) and an absence of risk adjustment in the stars calculation.

Overview of Nursing Home Star Assignments

Source : Design For Nursing Home Compare Five-Star Quality Rating System 2018 ⁵

It is important here to note that there can be no nationally consistent process for assigning star ratings, since the most significant contributors to the ratings are the state-level inspections, which are conducted by separate sets of inspectors within each state, using protocols for which there is no national standard.

While the reporting around staffing levels could lead the casual reader to believe that staffing levels had the largest impact on star rankings – they do not. Nursing home “overall” stars generated by the state-level inspections can only (potentially) be raised by one if staffing ratios warrant 4 or 5 stars or reduced by 1 if the staffing ratio is 1 star. The quality domain has a similarly small impact on overall stars with compare stars being largely driven by health inspection rankings and those rankings being state specific⁶.

The last point was emphasized in a letter to CMS penned by Congresswoman Jan Schakowsky (IL) and Senator Richard Blumenthal (CT). In their July 31^st letter to CMS:

“In addition, the complexity of your rating standard, updated as recently as July of 2018, continues to trouble me. Your guide allows for adjustments based on a number of factors, ultimately distorting the standardization of the ratings that facilities receive. It is nearly impossible to compare nursing homes across states on an equal standard.“⁷

The need to compare the quality of nursing home care across states brings us to our recent article, “Nursing Home Compare Star Rankings and the Variation in Potentially Preventable Emergency Department Visits and Hospital Admissions.”⁸ In the article, we compare nursing home performance across states using two outcomes measures: Potentially preventable admissions (PPA) and potentially preventable emergency room visits (PPV). We find that there is large systemic variation both across and within states for both PPAs and PPVs driven by variation in the rates of encounters associated with various types of infections acquired in the nursing home. These rates have little correlation with the nursing home compare star rankings.

Routine measurement and monitoring of care quality coupled with providing easily interpreted indicators through which to select care are important functions undertaken by CMS. To be more meaningful, those indicators need to better measure the outcomes of nursing home care without the artificial boundary of state lines.

Richard Fuller, MS, is an economist with 3M Clinical and Economic Research.

Norbert Goldfield, MD, is former medical director for 3M Clinical and Economic Research.

John S. Hughes, MD, is Professor of Medicine, Yale School of Medicine.

References

1. Office of Public and Intergovernmental Affairs. VA Extends Record of Transparency with First-Ever Posting of Annual Nursing Home Ratings. Published 2018.
2. Estes A. Secret VA nursing-home ratings hid poor quality care from public. Boston Globe. Published June 17, 2018.
3. Centers for Medicare & Medicaid Services. Transition to Payroll-Based Journal (PBJ) Staffing Measures on the Nursing Home Compare tool on Medicare.gov and the Five Star Quality Rating System. Center for Clinical Standards and Quality/Quality, Safety and Oversight Group. Published 2018.
4. Rau J, Lucas E. Medicare Slashes Star Ratings for Staffing at 1 in 11 Nursing Homes. New York Times. Published July 27, 2018.
5. Centers for Medicare & Medicaid Services. Design For Nursing Home Compare Five-Star Quality Rating System. Technical Users Guide. Published 2018.
6. Centers for Medicare & Medicaid Services. Fact Sheet: Nursing Home Compare Five-Star Quality Rating System. Published 2018.
7. Schakowsky J, Blumenthal R. Schakowsky & Blumenthal Urge CMS to Improve Quality Rating System for Nursing Homes. Published 2018. Accessed August 8, 2018.
8. Fuller RL, Goldfield NI, Hughes JS, McCullough EC. Nursing Home Compare Star Rankings and the Variation in Potentially Preventable Emergency Department Visits and Hospital Admissions. Popul Health Manag. July 2018:pop.2018.0065. doi:10.1089/pop.2018.0065.