Risk-adjustment should be about cohorts not individuals

June 11, 2018 / By Richard Fuller, MS

Viewing risk-adjustment as a process of balancing financial risk across defined patient cohorts, rather than the risk posed by an individual, is a position statement rather than an accepted norm. Risk-adjustment models can be built from the individual up by selecting and combining risk factors (e.g. elevated cholesterol or blood pressure) applicable to the whole population, or instead by first identifying discrete cohorts from within the population and identifying risk factors relative to them. The former is a structure more closely associated with regression-based risk-adjustment models, while the latter is more the domain of clinical categorical models. This is a topic we in the clinical and economic research team tend to deal with a lot.

The most unambiguous argument we present in favor of categorical models (cohort-based) over models derived by aggregating individual risk factors is the ability to communicate. The ability to define patient groups based on their enrollment type, the events that drive their costs and outcomes, the providers that treat them, or other common factors (such as socioeconomic status) that affect outcomes and costs, permits the creation of an effective management tool. To manage effectively, we would contend, requires examination of broad patterns of occurrences rather than individual instances.

But there are properties other than managing enrollees that are important attributes for risk-adjustment. For those worried about access to care for all individuals, particularly those in vulnerable subgroups, it is necessary to examine the actual payments that result from a risk-adjustment method compared to the costs incurred (relative profitability). As with others engaged in providing healthcare services, treating every health plan’s motivation as profit maximizing is far too simplistic. However, even a health plan that has a mission to serve vulnerable populations cannot afford to completely ignore the bottom line. It should therefore be of concern if risk-adjustment provides windfall profits for plans that are able to exclude easily identifiable (and typically vulnerable) population subgroups, not least as this means that other plans will receive actuarial losses from their enrollee mix. Plan financing is a zero sum game.

So with this backdrop, and acknowledging our bias, we are taking the opportunity to highlight a paper on this topic put forth by Sherri Rose and Tom McGuire at Harvard¹. The purpose of the article (and this blog) is not to list failings in existing risk-adjustment models, but rather to highlight methodological choices, particularly in evaluation, that have real world impacts. The authors step through a scenario in which they offer different outcomes for a risk-adjustment model depending on the statistical choices made. Model developers may look at overall R² (based upon the matching of cost and payment across all individuals), or parsimony (highest R² for fewest variables), or subgroup maximization (maximizing payment accuracy for a predefined subgroup). The approaches have different consequences for the final model – where consequences are quantified using a “net compensation” model for a defined patient population, which within the paper is examined using those with mental health and substance abuse conditions.  The statistical assessments of the risk-adjustment models are similar to those conducted by the Society of Actuaries in their periodic reviews². We mention this because actuaries are key to signing off on the fairness and accuracy of risk-adjustment models used to pay health plans and therefore how they assess the properties of risk-adjustment models plays a pivotal role in their use.

One thing the authors leave us with, and we would echo, is that when risk-adjustment models are used to define performance or allocate resources across distinct patient or population subgroups, it is essential that the impacts upon those subgroups be assessed. Our most recent contribution to this discussion was made a couple of years ago in our article, “Comparison of the Properties of Regression and Categorical Risk-Adjustment Models”³. In the article we state:

“Evaluating the accuracy of a model in matching payment to cost, or risk to likelihood of an outcome, has a tendency to rely upon an R² comparison of global fit. Although this may work with comparing overall historical patterns, it is essential to compare how well the model fits for subsections of a population—such as those with a particular chronic condition, or from a particular location, or with particular challenges or other identifiable risk factors. It should be assumed that those paid to take on risk will look for ways to minimize their risk relative to their payment and will adopt strategies to do so, making this more complex analysis more important than the overall measure of global fit usually utilized to compare models.“

The Rose/McGuire paper offers a reminder of the importance of focusing on the impacts of modelling decisions, with an attendant need for transparency and review, to avoid inadvertent harm for identifiable patient cohorts. Another aspect of this that goes unaddressed by the authors is the effect of using prospective (predictive) rather than concurrent classification as the basis of plan payment. This is a topic we have covered in a previous blog so I will not be redundant here. But the timing of computing risk scores and the projection of future cost can lead to significant bias against diseases such as cancer that have large costs close to the time of their detection or disease cohorts such as those with Mental Health conditions that tend to have periods of cyclical intensity. A clear example of this is given in a recent thought piece on ACOs put out by the Center for Healthcare Quality and Payment Reform⁴ in which a new diagnosis of lung cancer can add $100,000 to patient spending in the current year with risk scores impacted in a subsequent year. Individuals within cohorts with high initial treatment costs or with large cyclical costs are likely to inflict actuarial losses upon the plans in which they are enrolled (i.e. present elevated risk). In the world of insurance riskier cohorts are less attractive – a needless artifact of the prevailing risk-adjustment model.

Richard Fuller, MS, is an economist with 3M Clinical and Economic Research.

References

1. Rose S, McGuire TG. Statistical Fit and Algorithmic Fairness in Risk Adjustment for Health Policy.; 2018. https://arxiv.org/abs/1803.05513.
2. Hileman G, Steele S. Accuracy of Claims Based Risk Scoring Models.; 2016. https://www.soa.org/research-reports/2016/2016-accuracy-claims-based-risk-scoring-models/.
3. Fuller RL, Averill RF, Muldoon JH, Hughes JS. Comparison of the Properties of Regression and Categorical Risk-Adjustment Models. J Ambul Care Manage. 39(2):157-165. doi:10.1097/JAC.0000000000000135.
4. Miller H. HOW TO FIX THE MEDICARE SHARED SAVINGS PROGRAM.; 2018. http://www.chqpr.org/downloads/How_to_Fix_the_Medicare_Shared_Savings_Program.pdf.