Predictive models and population health – A clinical point of view (part two)

Feb. 19, 2016 / By Norbert Goldfield, MD

Predicting the behavior of human beings is hard – it is hard enough to figure out how to predict the behavior of our loved ones, let alone patients with whom we spend a total of a few minutes on an episodic basis. Yet, healthcare companies and pundits are promising us all sorts of ability to identify patients who are at risk for certain events such as preventable emergency room visits or hospital complications. This blog, part two of a series, provides a clinical point of view on the strengths and limitations of predictive modeling.

In examining clinical issues pertaining to predictive modeling, it is critical to be clear on the answers to the following questions related to model design:

1. Are we trying to predict the behavior of an individual patient or a group of patients?
2. What is the time period that one is trying to predict? Are we trying to predict what is going to happen in the next hour to a hospitalized patient, or are we trying to predict preventable events or complications that may occur to an individual patient over the coming year?
3. What type of data is available to “populate” the predictive model? Will claims data be used? What types of claims data? Does it, for example, include functional health status? Or will information drawn from an electronic health record or patient-derived information?
4. Are the results of the predictive model a complement to health professional decision-making or are the results used immediately for patient intervention?
5. What outcome is the predictive model trying to predict? Costs, mortality or morbidity?

I help to develop the clinical logic that populates models that, in turn, try to describe the clinical burden of human beings – for a number of dependent variables and for different time periods. I look at the development of these clinical models through a belief system that prefers to see healthcare delivery as a noble calling, but appreciate the fact that I am an anomaly and that many consider health care as a purely economic activity.¹ As a consequence, I try to assist in developing clinical models that can be used for both predicting cost as well as a clinical outcome such as complications. Clearly, the decisions on what to include for each of these two dependent variables might be different; however, I have found the keys to success are to build in as much redundancy as possible, and focus on the clinical outcome in question. I focus on the clinical outcome with the hope that clinicians will use the information to improve care, which, in most situations, will result in lower cost. (I should note, however, that this is not the situation in all cases, particularly among low-income populations that are underserved.) Development of categorical or rules-based models as described in previous blogs² facilitates building in redundancy and allows for transparency (even if detailed) in the clinical details of the model. In the final analysis, as federal policymakers have amply documented, the clinical models such as DRGs have worked because of the fact that clinicians can examine the logic and suggest changes.³

Yet, as Richard Fuller and I pointed out in part one of this series, models that try to predict a relatively speaking distant future (such as the coming year, not the coming hour(s) during a hospital stay) for an individual is fraught with statistical and clinical difficulty. I accept the fact that the “market” is demanding predictive models at the individual level. There are many commercial models that purport to confidently predict the individual level an outcome such as readmission at a distant future such as thirty days. I don’t believe it.

Instead, I am confident that the development of retrospective models as described in our last blog for, at least initially, groups of patients, is what makes sense clinically and will eventually prevail. These retrospective models need to continuously build on the ever-expanding types and sources of data available to populate these models. In particular, Richard and I, together with the rest of the Clinical and Economic Research team at 3M HIS, believe that we’ve identified the vast majority of what is potentially preventable in healthcare delivery. Granted, this detailed listing with many exclusions will continuously evolve – as all of medicine continuously evolves – hopefully for the better. Detailed risk adjustment of the individual’s burden of illness is nothing short of critical for fair comparison of results – either at the individual health professional level or at the level of the healthcare system. Detailed risk adjustment is all the more important today as payers are actively trying to shift financial risk for providing care for vulnerable and/or individuals with multiple burdensome illnesses. If these models are to be ultimately useful, health professionals need to dig deeply into the results of the model and use this information to enhance patient outcomes, and in fact, save lives not just save money.

Norbert Goldfield, MD, is medical director for 3M Clinical and Economic Research.

¹See the New York Times from February 16, 2016, “Founded for the Poor, Mass General Looks to the Wealthy.” http://www.nytimes.com/2016/02/14/your-money/founded-for-the-poor-mass-general-looks-to-the-wealthy.html?login=email&ref=health&_r=0).

²Clinical Categorical vs. Regression Based: Understanding classification system fundamentals, September 23, 2013. Blog by Norbert Goldfield, MD, Richard Fuller.

³https://www.gpo.gov/fdsys/pkg/FR-2001-09-07/pdf/01-22475.pdf. “The success of any payment system that is predicated on providing incentives for cost control is almost totally dependent on the effectiveness with which the incentives are communicated… Central to the success of the Medicare inpatient hospital prospective payment system is that DRGs have remained a clinical description of why the patient required hospitalization.” (Federal Register, Vol. 66, No. 174/Friday, September 7, 2001/Rules and Regulations) p 46904