A few months ago we demoed Caremaze to a large group at a health system, and the entire conversation collapsed onto one feature: the AI agent that makes phone calls. I understand why. It is vivid, it is new, and watching software complete a call that a human used to make is the kind of thing people want to see twice. But it pulled the whole room away from the question that actually matters, which is not how much AI can do in a hospital but what it should do. After hundreds of hours on hospital floors, we have a clear answer to that question, and the answer is a boundary rather than a capability.

The part everyone wants to talk about matters least

Every AI pitch a hospital executive sits through right now leads with the most impressive thing the model can do. I have watched the same dynamic from the other side of the table, and the demo moment that draws the most attention is reliably the one that matters least to outcomes. The phone call is just a phone call. Case managers make dozens of them a day, and nobody has ever argued that dialing a transportation agency is the highest expression of clinical skill. What the room should be asking is which calls the software makes, which it never makes, and who decides what happens with the information that comes back.

A physician leader at a large academic medical center put his finger on it during a recent conversation. He had been watching the product populate a discharge checklist and said, “You can imagine that the real service you’re offering is, if you had a really good reader who was really good at Excel and just didn’t make mistakes, they could be populating that checklist themselves. It’s just they can’t keep up.” That is exactly right, and what he was describing was not a robot making decisions but a tireless reader and tracker working underneath the people who do.

The line we draw

I give the same answer in every one of these conversations: AI is great at doing the dishes. You do not want it running the kitchen.

The dishes are the transactional work of discharge coordination. Gathering information, transmitting information, chasing the status of an order, scheduling and rescheduling, confirming that what was supposed to happen actually happened. The test I use is whether the task requires a license. Calling a third party to exchange information does not. You are not exercising judgment, you are moving facts from one place to another, and that is work an AI agent can hold.

The kitchen is everything else. What this patient needs, in what order, whether a given option is clinically appropriate, when the plan has to change, and every conversation with a patient or a family. That work stays with the care team, entirely. We have no interest in being in that area, and I say that not as a compliance posture but as a design principle. One health-system executive described discharge tasks to me as “co-owned, which means they’re owned by no one.” You do not fix that by letting software take ownership of decisions. You fix it with software that makes ownership visible and takes the legwork off the owner’s plate, so the person with the license can actually exercise it.

Below license is not the same as unimportant

This is where the dishes metaphor can go wrong, so let me be precise about it. The work AI takes over is necessary work that somebody has to do, and the problem is who has been doing it.

In the hospitals where I have spent time, I have watched case managers and social workers spend 60 to 70 percent of their time on this transactional layer. These are experienced, mission-driven clinicians, and they spend most of their day as the connective tissue between fax machines, voicemail systems, and portals. Somebody has to call the transportation agency to move a pickup from noon to 2 p.m. But is that worth 45 minutes of a nurse case manager’s time? The same physician leader described the human cost of running coordination this way: “They’re tracking a lot, they’re fallible, they’re human. We probably don’t have enough of them. Every time they’re too busy looking at Mr. Davis’s stuff, that’s why Miss Jones’s stuff gets backed up three or four hours.” Those hours add up, because the home care agency that receives ten patients’ orders at 4:01 p.m. instead of 3:59 delivers tomorrow instead of today.

I wrote about what case managers told me they actually want in an earlier post, and the short version is that none of them asked for help with judgment. They asked for relief from the work below their license so they could do the job they trained for. The frustration belongs to the system that wastes their expertise, not to the people, and any automation strategy that reads as disrespect for that expertise will fail on contact with the floor.

What this looks like in practice

The concrete cases make the boundary easy to see. A case manager at an urban academic medical center needs a cardiologist in a rural area for a patient who lives far outside the referral network. The agent finds candidates, verifies availability, verifies insurance, and brings the information back. The case manager decides whether the referral is appropriate, because her license was never needed for searching directories and sitting on hold with an insurance line. It is needed for the decision.

Skilled nursing placement works the same way. The status quo for a hard-to-place patient is three or four calls a day, two of which go to voicemail, and then these get returned while the case manager is busy with someone else. One leader told me his team relearns the same market with every staffing change: “I have to reinvent the wheel with every new case manager that touches that patient.” Instead, the agent can spin up a couple hundred intake calls at once and return a same-morning read on who has a bed. The placement decision, with everything it weighs about the patient and the family, stays human.

Or take PICC lines. A doctor orders the placement, the order joins thirty others in a queue, and nobody tracks whether it ever becomes a scheduled procedure and then a completed one. Today, that prioritization lives in one coordinator’s head, and as the same leader put it, “maybe they’re good at that and maybe they’re not so good at it.” With AI, the system watches every order, flags the patient for whom this is the last barrier to discharge, and surfaces it to the team at 8 a.m. The division of labor is the same in all three cases: the AI gathers, verifies, and tracks so that the clinician can decide.

Where the line gets blurry

I will admit the boundary is cleaner in a sentence than on a ward. The same physician leader walked me through his unit’s discharge chain: the patient cannot leave until suction equipment is waiting at home, the home care agency will not deliver it until the nurse signs off on trach care teaching, and the nurse cannot sign off until the family completes education at the bedside. Scheduling a delivery sounds transactional, but in that chain it sits two steps from a clinical competency assessment. Some tasks look like dishes and carry clinical risk if they are done wrong or out of order.

That is exactly why a human stays in command of the workflow. The right analogy is autopilot. It flies the routine cruising portion of nearly every commercial flight, but the pilot remains in command, handles the landing, and takes over the moment anything requires judgment. Nobody describes that arrangement as the autopilot replacing the pilot. In our case, the discharge goals the system works from come from a library of standards that the hospital reviews and edits before anything goes live, and the system proposes while the staff approves. The same leader played the workflow back to me after seeing it: “It flashes up and says, we think Miss Davis needs home oxygen. Is this correct?” Only after the case manager says yes does the plan expand into tasks the system starts tracking and chasing. When a task sits in that ambiguous zone, it goes to a person, every time.

A clear boundary is what makes adoption safe

Clinicians have every reason to be skeptical of one more piece of software, and the skepticism extends to the technology itself. A medical director told me flatly, “When I get a phone call and I think it’s AI, I hang up.” I take that as a rational response to a market full of tools that blur the line between assisting clinicians and replacing their judgment. You do not answer that skepticism with a better demo. You answer it with a boundary the care team can verify for themselves: every call recorded and reviewable, every proposed plan approved by staff before it goes live, every decision traceable to a person with a license.

This is also why the same physician leader who pushed us hardest ended the conversation by describing the product as “a force multiplier for your case managers.” That phrase only makes sense if the case manager remains the unit of value. What we have built makes their expertise go further by clearing the transactional layer out from under it, which is what health systems mean when they talk about practicing at the top of license. As he put it, “you don’t know what you don’t know until you start looking,” and looking is precisely the work software should be doing around the clock. We compress the whole philosophy into four words on our website, “AI suggests, care teams decide,” and the technology is only safe to adopt because that line is explicit. We would rather be trusted for where we stop than admired for how much we automate.

If you are trying to figure out which parts of discharge are safe to hand off and which are not, that is the conversation we like having. Reach out anytime.

Pierre-Jean Cobut, CEO & Founder of Caremaze | LinkedIn

Frequently Asked Questions