Hospitals Are Deploying AI Like Crazy. We Still Don’t Know If It Helps Patients.

I don’t need to tell you that AI is everywhere.

Or that it’s showing up in hospitals at a pace that surprises even the people building it. Doctors use AI scribes to take notes during appointments. Algorithms trawl through electronic health records to flag patients who might need extra attention. Machine learning models interpret chest x-rays and lab results.

A lot of studies show these tools can be accurate. But there’s a much bigger question that’s getting skipped: does using them actually make patients healthier?

We don’t have a good answer. And that’s exactly the point Jenna Wiens, a computer scientist at the University of Michigan, and Anna Goldenberg from the University of Toronto make in a paper published this week in Nature Medicine.

Wiens told me she spent the first decade of her career trying to pitch AI to clinicians. It was an uphill battle. Then, over the last few years, something shifted. “A switch flipped,” she says. Suddenly healthcare providers are not just interested—they’re deploying these tools fast. The problem is they’re not rigorously checking whether they work in practice.

Take ambient AI scribes. These tools listen to doctor-patient conversations, then transcribe and summarize them. Multiple products are already widely adopted. A staffer at a major New York medical center told me a few months ago that doctors are “overjoyed”—the tech lets them focus on patients instead of typing notes, and early studies do show reduced burnout.

That’s great. But what about the actual health outcomes? “Researchers have evaluated provider or clinician and patient satisfaction, but not really how these tools are affecting clinical decision-making,” Wiens says. “We just don’t know.”

The same problem shows up across the board. Some AI tools predict a patient’s health trajectory. Others recommend treatments. They’re all designed to make care more effective and efficient. But accuracy alone doesn’t guarantee better outcomes.

Consider a tool that speeds up chest x-ray interpretation. Even if it’s spot-on, how much will a doctor actually rely on it? How does it change the way they interact with the patient or recommend follow-up? And what does that mean for the patient in the end? The answers could vary by hospital, department, or even by how experienced the doctor is.

Wiens points to research on AI in education, which suggests these tools can change how people cognitively process information. Could an AI scribe subtly alter how a doctor processes what a patient says? Could it affect how a medical student learns to think about patient data? “We like things that save us time, but we have to think about the unintended consequences,” she says.

A study published in January 2025 by Paige Nong at the University of Minnesota found that around 65% of US hospitals used AI-assisted predictive tools. Of those, only two-thirds evaluated their accuracy. Even fewer checked for bias. The numbers have likely gone up since then, but the evaluation gap hasn’t closed.

Wiens doesn’t want to stop AI adoption. She just wants to see real assessment. “I do believe in the potential of AI to really improve clinical care,” she says. “I have to believe that in the future it’s not all AI or no AI. It’s somewhere in between.”

Right now we’re somewhere in between, but we’re flying blind. That’s the part that needs to change.