AI in healthcare: evidence, workflow and the limits of automation
A practical evidence guide to AI in healthcare, separating authorised products and useful workflow support from broad claims that algorithms can replace clinical judgement.
Last reviewed 2026-07-25
Where is AI already used in healthcare?
AI is already used across medical imaging, decision support, monitoring, workflow prioritisation, documentation and other clinical or operational tasks. The US FDA said in January 2025 that it had authorised more than 1,000 AI-enabled medical devices through established premarket pathways. That is evidence of real deployment, but authorisation of a device is not evidence that every use improves outcomes in every setting.
What should healthcare organisations evaluate before adoption?
Evaluate the intended population, data used to develop and validate the system, clinical workflow, failure modes, human oversight, performance across relevant groups, monitoring after deployment and what happens when the system or upstream data changes. A technically strong model can still be a poor clinical product if it adds alerts, shifts responsibility ambiguously or performs differently in the local patient population.
Can AI replace clinical judgement?
Some systems can automate tightly defined tasks, but broad clinical judgement combines evidence, context, patient preference, uncertainty and responsibility. The better question is where automation can reduce burden or improve consistency while keeping accountable human review at the points that matter. “Human in the loop” is not magic either: the human needs time, information and authority to challenge an output rather than merely rubber-stamp it.
What is the non-obvious implementation risk?
Workflow failure can matter as much as model failure. A system that is accurate in a test set can still cause harm if staff misunderstand its scope, if alerts arrive at the wrong time, if data drift goes unnoticed or if responsibility becomes blurred. Healthcare AI evaluation therefore needs operational evidence alongside accuracy metrics, including how the tool changes real decisions and workload.
What does the current evidence say?
- The FDA said in January 2025 that it had authorised more than 1,000 AI-enabled medical devices through established premarket pathways. Source
Limitations
US FDA authorisation data does not describe UK regulatory status or prove clinical benefit in a specific NHS or private-care setting. Device counts also depend on the regulator’s categorisation and change over time.
A counterpoint worth keeping
More predictive accuracy can still make a service worse if the tool creates alert fatigue, slows decisions or shifts work to already overloaded staff. Adoption is a systems problem, not just a model benchmark.
Sources and provenance
- US Food and Drug Administration · FDA Issues Comprehensive Draft Guidance for Developers of Artificial Intelligence-Enabled Medical Devices · 2025-01-06 · Primary source
- Information Commissioner’s Office · Guidance on AI and data protection · 2025-06-19 · Primary source