Epic UGM: AI ambitions meet health system reality

If last year's Epic User Group Meeting felt like science fiction, this year's felt distinctly down-to-earth.

At least, that was my impression as the futuristic sets and big-picture visions of AI transformation gave way to a theme of Midwestern hospitality at Epic's sprawling Verona, Wisconsin, campus this week.

Don't mistake the approachable packaging for a retreat from the company's AI ambitions—quite the opposite. Epic's message this year was that much of what sounded futuristic 12 months ago is getting closer to everyday work.

One skit captured that transition particularly well. In an Inside Out-esque portrayal of an AI agent at work, Epic employees played the different components operating behind the scenes. Each enthusiastically declared, "This is my favorite part!" before doing their job and, eventually, pressing a giant button to send out the query.

It was characteristically Epic: playful, elaborate, a little strange.

But it also captured two conversations I kept hearing throughout the general session and in the halls afterward.

First, we're talking less about what AI might someday do and more about what it is doing today.

Second, proving that the technology works doesn't necessarily mean health systems are ready to use all of it. Epic repeatedly encouraged customers to turn on more features and make better use of the products already at their fingertips. Some health system leaders, meanwhile, are still figuring out whether they have the people and infrastructure to safely manage everything that's becoming available.

The platform gives health systems tools to build and adapt agentic workflows for their own needs. Instead of waiting for Epic to develop a capability or buying another product from another vendor, they can increasingly build it themselves.

Inova Health has 70 AI features in production, about 20 percent of which are agentic, Chief Data and AI Officer Jon McManus told me at UGM. Its Agent Factory work on infection prevention has cut chart-review time 15-fold. During a measles exposure, McManus said, that can shrink a process that might otherwise eat up much of the 72-hour window for intervention to about six hours.

ECU Health offered another example. Vice President of Clinical Operations Jacob Parrish said an Agent Factory workflow has helped drive an 86 percent increase in transfers landing at regional hospitals by prompting transfer nurses to consider alternatives to automatically sending patients to the medical center.

"When you get into the actual operational value, it's there," Parrish told me.

That's the question Dr. Hasan Ahmad, associate CMIO at Parkview Health, left UGM asking.

Agentic systems aren't "build it and forget it," Ahmad told me. Models change. Inputs change. Prompts change. Outputs can change with them. And somebody has to watch for that degradation, or model drift.

"These agents are going to be built with Epic, but eventually, organizations are going to own the maintenance and monitoring," Ahmad said. "That's going to require additional resources, time, data scientists, informaticists, and governance infrastructure — and some organizations are not going to be able to do that. That's the part of the discussion I would like to see more of."

Parkview has identified potential Agent Factory use cases. But Ahmad isn't ready to separate the excitement around building them from the realities of operating them.

"We've identified candidate use cases, but we're still going to need more clarity from Epic on what the operational model is going to look like," he said. "I think Epic is interested in developing these concepts and ideas. However, building something is different from running it inside a health system."

Epic isn't ignoring that problem. Nurse Executive Johnston Thayer told me the company is building an Evaluation Suite that will allow health systems to evaluate agents before they go live, monitor whether they continue operating within defined guardrails afterward and measure whether they're actually producing the intended outcomes.

Still, the general session's lack of detail here left several executives I spoke with on Wednesday—including Ahmad—with questions.

Then there's another group watching all of this closely: Epic's partners and competitors.

Last year, I spent much of UGM asking what Epic's expansion into ambient documentation and other AI capabilities meant for the companies already selling those products. Health tech leaders weren't exactly unconcerned. As one CEO told me at the time, "it's really tough to go against Epic with something they do first-party."

This year, I put a version of that question to Abridge CEO Dr. Shiv Rao. How does Epic's push into agents and the expansion of Art affect his strategy?

Rao said Abridge is confident it is "headed in a very different direction." The company continues to partner with Epic, he noted, while also working across EHRs and with payers and life sciences companies. Even where their products may appear to compete, he expects health systems will see that "they're actually coming at these problems in a very different way."

That relationship is worth watching, especially because Epic spent plenty of time talking about community this year, too.

There was an emphasis on sharing features between health systems, announcements of more smaller platforms in the mold of Orchard and the introduction of Epic Rangers to help organizations hone the platform one-on-one. Increasingly, the company seems to want provide the infrastructure on which customers and partners can build, customize and share their own.

But quietly, there is another conversation happening behind that all-of-us messaging and the whimsical, cotton-candy facade. Just days before UGM, Reuters reported that the Federal Trade Commission is investigating Epic, including how the company grants or withholds access to data. Epic denied engaging in anticompetitive behavior and said its health system customers, not the company, control access to patient records.

I heard versions of that same question throughout the week, albeit in much less legalistic terms. As Epic gives health systems more tools to build, share and deploy their own technology, its customers and partners are also working out where Epic's responsibility ends and theirs begins. Who owns the data? Who owns the innovation built on top of it? Who is responsible for keeping an agent safe once it leaves the Factory floor?

What did you make of the UGM news? Email me at [email protected] and let me know.

And read on to the Pulse Check section for a closer look at Epic's Agent Factory, in practice.

At Epic's User Group Meeting, I sat down for a four-way interview to learn more about Agent Factory: how health systems are approaching development, governing new agents and recalculating the "build-versus-buy" equation as platforms like Epic give them more tools to build AI in-house.

Here's a portion of my interview with Johnston Thayer, chief nurse executive at Epic; Jon McManus, chief data and AI officer at Inova Health; Derek De Young, an Epic research and development leader working on Agent Factory; and Jacob Parrish, vice president, clinical operations at ECU Health.

Editor's Note: Responses have been lightly edited for length and clarity.

When you think about the kind of workflow that would benefit from an AI agent as opposed to another type of automation or technology, what is the differentiator? What types of workflows work best through an agent?

Johnston Thayer, Epic: One of the things that is very true about health care is that it's nuanced. There are a lot of processes that are deterministic in many regards, and so a simple set of if-then statements can handle that.

There's also a huge canvas of space where problems are much more nuanced than that, and it's not as simple as if this, then that. There needs to be some level of intelligence that is applied, whether that's referencing a policy or a protocol, bringing together and collating several different types of information to make a determination. When there is ambiguity, an agent allows you to create tools that can handle some of that ambiguity in ways that we couldn't prior to these technologies.

Jon, when you first started thinking about working in Agent Factory, which workflows did you think could benefit the most from this type of technology?

Jon McManus, Inova Health: It's probably less about roles and more about opportunities that are good candidates. We still want to practice safety and reliability in the decisions we make. We are a health system by nature, so when we think about AI, it has to be a good candidate for AI, and we have to think through: Is it safe and responsible to use AI to solve that problem?

In some ways, AI can be the riskiest choice we can make to solve a problem. We want to be thoughtful about that.

That's one of the reasons we tackled infection prevention, where we deal with large-scale exposures and needed to accelerate the ability to review those charts. Charts can be difficult because there's information in notes, scanned images, different types of media, as well as many discrete places in the Epic chart. Being able to bring all those things together and replicate what an infection preventionist would do to check a chart for an exposure event was a really good candidate.

Derek De Young, Epic: The cool thing about agents is it's not just what's traditionally talked about as a single-shot AI, where you feed it information, it processes it, thinks about it and then does an output. With an agent, that reasoning model also has access to tools that can interact with the system to get more data, search for more data or actually take action and set data into the system.

In this measles exposure, we didn't just have an agent blindly looking over a population. We had tools in Epic that could find the subset of patients that were actually needed. At that point, that's where the handoff happened to a nurse who is the skilled person who could do that actual chart review. That's where the AI can take over from that subset to drill down into what needs to be done and tee up recommendations for the nurse to take the next step.

This is called an Agent Factory, which makes me think of producing a lot in a short period of time. Do you envision organizations producing dozens or hundreds of agents, or is there some practical limit to how many you can manage?

Jon McManus: We have 70 AI features live at Inova right now in production. Roughly 20 percent are agentic in nature. We require a fusion team anytime we work on an agentic workflow where we have an ability to shape or make something.

We need someone who has that data engineering, data science, AI engineering pedigree, someone who can do the testing and evaluation work and technical stewardship. We pair that person with someone who's more clinical application or business application. And then we need that clinical operator or administrative operator to drive the business and operational responsibility of the problem we're trying to solve. We require those three roles every time we take on some type of agent workflow.

Derek De Young: I don't think we'll end up having thousands and thousands of bespoke agents that are fully custom built. I think we will have these core agents, these core agent harnesses, and we'll probably have thousands and thousands of skills that we're providing these agents long term, shared across the community, unique to an organization or released by Epic.

Jacob Parrish, ECU Health: We have a pretty robust governance structure in place. We call it the AIM Committee (AI Medicine). There are all kinds of AI initiatives happening outside of Epic-related AI initiatives, and then there are the ones that are with Epic.

The ones with Epic, I don't want to say are easier by any means, but they're a little less complex from a governance perspective because all the data is within the Epic environment. We've been able to scale Factory use cases pretty quickly through governance because we were piggybacking on our already established relationship, and that gave us a lot of comfort that everything was going to be done in the right way and ethically.

In terms of how many is too many, I don't think we can answer that yet because we don't know. We've not gotten to a point of saturation where we're like, well, we need to pause.

Johnston Thayer: As you think about a factory and the ability to produce things at scale safely, a key component is both process and quality checks. We're building a platform called the Evaluation Suite that allows for evaluation of your agent before it ever goes live in production. It also allows for monitoring after you go live to make sure that agents are still acting within the defined guardrails in the ways that you expect. Importantly, it allows you to define and measure outcomes: Is it actually making the impact that we want it to?

A couple of years ago, I was hearing a lot of concerns about agents interacting with patients autonomously without clear oversight. Has that changed? Do you have patient-facing agents?

Jon McManus: We do have patient-facing AI, and we plan to expand that. As a health system, you have to take a position on consent, disclosure and transparency. That's very important.

It's also very important that when AI is used, it is in a support position. It's not diagnosing. It's not making recommendations that licensed individuals are required to make. It is important to have all those safeguards in place.

Jacob Parrish: Our tagline in all that is decision support, not decision replacement. A human still needs to make that decision.

How does Agent Factory change the build-versus-buy calculus? Are there tools you previously would have bought that you feel like you can now build yourself?

Jon McManus: What we see in Epic is more of a full-service product built from a trusted position. We see the right protections in place. We see the right guardrails. We see the right testing and evaluation suite. These are typically services that you have to figure out as a company to stand up around the build tool.

We see Agent Factory as probably a growing ratio of our agent-related build because it's a full-service product. We plan to use it quite a bit.

Jacob Parrish: From an organizational perspective, we have the philosophy of Epic first. Can Epic solve this problem, whatever that problem is? If they can, great. Let's dive in on that. If they can't, then we go and look for other options.

When you get into the actual operational value, it's there. We've seen an 86% increase in us landing transfers in our regional hospitals because the logic that we built within Factory is prompting our transfer nurses to maybe look over here. They've always had that knowledge, but when they get in a pinch, or the phones are ringing, or it's chaotic, the easiest thing is to default to the medical center. By having this served up in front of them, the operational value is pretty significant.

Jon McManus: We were able to improve chart review time 15x. If you think about a measles exposure, you have 72 hours to respond with MMR for immunocompromised patients. If it takes six to eight minutes to manually review each chart and you have a couple hundred patient exposure event, you run out of 72 hours very, very quickly.

When you can introduce an agentic assist where it is taking seconds to do those chart reviews, you reduce the time to complete that work to about six hours. Now you have the remainder of the time to try and intervene with those patients. These are just our first use cases.