CAC Exact: A new CAC system performance metric

Nov. 9, 2016 / By Clarissa George

What percentage of automation is completely correct in a computer-assisted coding (CAC) system? Or, in other words, how many visits does a CAC system get exactly correct? The answer to this question is what we call “CAC Exact,” a concept that the business intelligence team at 3M HIS has been exploring and measuring. After assessing the CAC system’s data, we found measuring CAC Exact visits can give coding managers and HIM directors another tool to evaluate a CAC system – especially an outpatient CAC engine.

CAC Exact Visits

CAC Exact is a measure of visits that are auto-suggested exactly and correctly. This means the engine saw the documents for the visit and suggested the correct codes precisely – the engine didn’t over-suggest nor did it under-suggest any codes. In engine measurement terms, recall and precision were 100 percent. The measurement of CAC Exact visits is the percentage of visits that were CAC Exact out of all visits. CAC Exact can be measured for an enterprise, facility, etc.

The reason the measurement works best for outpatient encounters is because outpatient cases are simpler and less unique from one facility or organization to another. Inpatient cases are more complex and measuring how often a CAC system gets a visit exactly right doesn’t make a lot of sense as this figure will be small due to the variability in inpatient cases. For outpatient visits, however, this measurement offers a simplified way to see how much impact a CAC system has or could have for a facility or hospital.

To illustrate the use and effectiveness of CAC Exact, let’s take a closer look at some hospitals and their outpatient data, shown below in Figure 1.1.

Figure 1.1 Sept. 2016 Data
Hospitals CAC Exact Outpatient Data

In Figure 1.1 we see that each of the hospitals had some visits that were CAC Exact. Though the percentage of the visits vary, we can see that the hospitals shown had somewhere between 5 and 18 percent of their visits that the Outpatient (OP) CAC system got exactly right. What does this tell us? It tells us the value of the CAC system in another way. The percentage of the OP CAC Exact indicates how much the CAC system is helping. In this case (and without factoring into account anything else that assists coders), Hospital System C’s CAC system suggested OP visits exactly correct 9.24 percent of the time (in September 2016)!

Note: One could take the number of minutes per chart for the visits that were CAC Exact and show how much quicker these visits were coded compared to each Hospital System’s overall minutes. However, the CAC Exact visits often consist of only a few codes, so comparing them to overall minutes per chart can distort the picture.

You might ask “So how do I turn that into hours saved or hours I could save?” or “How do I turn this into a measure of how much value I am getting from the CAC System?” Let’s take a look at Figure 1.2 below to answer these questions.

Figure 1.2 Sept. 2016 Data
Hospitals CAC Exact Outpatient Data Including Coder Acceptance

In Figure 1.2 I have added the CAC and Manual methods for the visits that are CAC Exact. Here we are focused ONLY on visits that are CAC Exact. We can see how Hospital System A – D entered in the codes for these visits (CAC Methods vs Manual Methods). Looking at Hospital System B, we can see that the average minutes per chart for CAC Exact visits was 0.94 minutes. To get Estimated hours spent coding manually per month we took (OP CAC Exact Count x Minutes per Chart for Exact Visit x Manual Methods)/60. This gives the estimated hours spent coding manually per month when the coder really didn’t need to.

I hope this illustration helps to (1) show the value of seeing, at a glance, how much automation the product is doing for you and (2) how coders may be able to decrease the time it takes to code (and save all those hours) to spend their time doing more beneficial things – whether that be more complicated cases or just more cases in general. Understanding measures such as CAC Exact will become increasingly important as CAC Systems mature to the point of delivering true automation where possible.

Clarissa George is a business intelligence specialist at 3M Health Information Systems.