Moving from “old thinking” comparisons of claims-based vs. chart-based methods for quality improvement

June 20, 2016 / By Norbert Goldfield, MD, Richard Fuller, MS

A recent article in the journal Medical Care examines the validity of AHRQ Patient Safety Indicators (PSIs) and CMS Hospital-acquired Conditions (HACs) given the growing reliance of these measures for both public reporting and pay-for-performance initiatives. We believe that this article, Validity of the Agency for Health Care Research and Quality Patient Safety Indicators and the Centers for Medicare and Medicaid Hospital-Acquired Conditions: A Systematic Review and Meta-Analysis (Winters), presents two major challenges, one of which may have significant policy implications.

First, there are significant methodologic challenges impacting the authors’ conclusions, which we will deal with separately in other publications.  In this blog, we will comment on the second challenge—the potential policy implications of this article given that one of the co-authors, Peter Pronovost, PhD, MD, is a senior person in the quality improvement field.  Fundamentally, we believe that the authors are asking the wrong question in this article. Instead of simply evaluating the validity of chart- vs. claims-based identification of hospital complications, the authors should have examined the impact on hospital complications of the dramatic advances in computer technology linking claims data with other data elements, the increased clinical information available from claims, and the impact on health outcomes in states that have implemented claims-based identification of hospital complications. The Medical Care article also makes general and incorrect statements on an issue we have extensively discussed in these blogs; that is, how one implements the financial incentives is just as important as the complications themselves.

To summarize the article by paraphrasing from the abstract: “AHRQ Patient Safety Indicators (PSIs) and CMS Hospital-acquired Conditions (HACs) are increasingly being used for pay-for-performance and public reporting despite concerns over their validity.  The authors searched MEDLINE and the literature from January 1, 1990 through January 14, 2015, for studies that addressed the validity of the HAC measures and PSIs. Only five measures had sufficient data for pooled meta-analysis. One PSI, 15 (Accidental Puncture and Laceration), met their threshold for validity but this result was weakened by considerable heterogeneity. Coding errors were the most common reasons for discrepancies between medical record review and administrative databases. POA modifiers may improve the validity of some measures. This systematic review found that there is limited validity for the PSI and HAC measures when measured against the reference standard of a medical chart review. Their use, as they currently exist, for public reporting and pay-for-performance, should be publicly reevaluated in light of these findings.”

In the literature review of earlier studies, the authors highlighted the absence of a present on admission (POA) modifier as being problematic. This has, of course, already been resolved by adding the field and requiring its use in routine reporting. Moreover, the advent of the 5010 837I claim transaction in 2012 expanded the number of secondary diagnosis and procedure codes reported from 9 and 6, respectively to 25 for both, meaning more detailed information is routinely captured by claim-based measures even before the transition to the detailed ICD-10 code set.² Taken together, it should be apparent that claims-based measures have improved and continue to improve rapidly.

The main issue, however, with significant associated policy implications, is the way in which the authors frame the research question.  In the 1990s there were many articles published comparing claims-based with chart-based approaches to measure quality of care. In the era before electronic medical records (EMRs), it would have been reasonable to make this stark comparison as the underpinning of an important article on quality improvement. That day has long gone. Today, we know from research across a number of states that when implementing financial incentives with a comprehensive set of potentially preventable hospital complications (instead of the small number of CMS hospital-acquired conditions, for example), complications decrease and, accordingly, lives have been saved.^1,4

Instead of conducting what we consider to be “old thinking” types of studies comparing claims- and chart-based methods, we would encourage researchers to refine the specifics of which claims-based complications are, in fact, potentially preventable and to specify exclusions for these complications. In part, we do this already working with clinicians in states that have implemented our approach to identifying complications. We also enhance claims data with additional data elements that are either coded in ICD-10 and/or linked with the EMR. Thus, for example, we have linked health status with claims data in our population health classification system. We should work together as a research community to either add chart-based data elements to claims-based classification systems (as occurs on a regular basis), or to insist that hospital complications need electronic linkages for claims and charts for specific data elements. Lastly, and just as importantly, the HAC program should be dramatically expanded to include a much wider range of potentially preventable complications.

As we have highlighted in previous blogs, there are many challenges in the current CMS implementation of the Hospital Acquired Condition Reduction Program (HACRP). The authors of the Medical Care article apparently do not share our belief that it is essential to look at rates (instead of maintaining that a particular complication was always preventable) and that the financial incentives of the CMS policy are poorly structured and discriminate against those hospitals that treat the sickest patients. We have just published a paper on this issue and we cannot understand the logic of CMS administrators in their apparent continuation of discriminatory financial incentives.³

Simply put, we strongly believe that inaccuracy in claims-based data as compared to chart-based information, given the expanded capacity to report detailed clinical information through claims, is the responsibility of the hospital. But, computerized solutions to this challenge do exist. The financial incentives that CMS implemented need to be fair and impact a comprehensive set of hospital complications. We should be working as a quality improvement and research community to implement these changes and continuously improve our ability to identify and decrease potentially preventable complications – wherever they may occur. 

Richard Fuller, MS, is an economist with 3M Clinical and Economic Research.

Norbert Goldfield, MD, is medical director for 3M Clinical and Economic Research.

References

¹Calikoglu, S., Murray, R., & Feeney, D. (2012). Hospital pay-for-performance programs in Maryland produced strong results, including reduced hospital-acquired conditions. Health Affairs (Project Hope), 31(12), 2649–58. http://doi.org/10.1377/hlthaff.2012.0357

²Centers for Medicare & Medicaid Services. Pub 100-04 Medicare Claims Processing;Transmittal 2028;Change Request 7004 (2010). Retrieved from https://www.cms.gov/Regulations-and-Guidance/Guidance/Transmittals/downloads/R2028CP.pdf

³Fuller, R. L., Goldfield, N. I., Averill, R. F., & Hughes, J. S. (2016). Is the CMS Hospital-Acquired Condition Reduction Program a Valid Measure of Hospital Performance? American Journal of Medical Quality : The Official Journal of the American College of Medical Quality. http://doi.org/10.1177/1062860616640883

⁴Patel, A., Rajkumar, R., Colmers, J. M., Kinzer, D., Conway, P. H., & Sharfstein, J. M. (2015). Maryland’s Global Hospital Budgets–Preliminary Results from an All-Payer Model. The New England Journal of Medicine, 373(20), 1899–901. http://doi.org/10.1056/NEJMp1508037

5Winters, B. D., Bharmal, A., Wilson, R. F., Zhang, A., Engineer, L., Defoe, D., … Pronovost, P. J. (2016). Validity of the Agency for Health Care Research and Quality Patient Safety Indicators and the Centers for Medicare and Medicaid Hospital-acquired Conditions: A Systematic Review and Meta-Analysis. Medical Care. http://doi.org/10.1097/MLR.0000000000000550