Machine learning (ML) is reshaping medical devices, offering faster diagnostics and more responsive care. But as these systems move from research into clinical use, their limitations—shortcut learning, automation bias, and regulatory gaps—demand careful scrutiny. A recent wave of developments, including the UK’s National Commission into the Regulation of AI in Healthcare, highlights the growing recognition that the impact of ML depends as much on governance as on technology.
ML models can analyze heart MRI scans in 20 seconds with 40% greater precision than human analysis, saving thousands of clinician days annually.
Shortcut learning—where models learn spurious correlations rather than true pathology—can lead to systematic misdiagnosis, as seen in chest X-ray AI tools for pneumothorax.
Automation bias, the tendency to uncritically accept machine output, is a major patient safety risk in clinical settings.
Edge AI processes data locally on devices, reducing bandwidth, latency, and privacy risks; the Neocam handheld cataract screener for newborns is a notable example.
The UK MHRA launched a National Commission on AI in Healthcare regulation in September 2025 to address these challenges.
Pattern Recognition: Where ML Excels
ML models can process large volumes of data faster than humans and, in some applications, with greater accuracy. With diagnostic demand rising and specialist staff shortages putting pressure on services, this capability can help clinicians manage workloads and reach decisions sooner.
For example, a British Heart Foundation–funded AI tool can analyze a heart MRI scan in just 20 seconds, detecting changes in heart structure and function with 40% greater precision than human analysis. It is estimated to save around 3,000 clinician days per year, freeing up specialists for more complex cases.
The Shortcut Problem
Impressive as these results are, they come with a crucial caveat: the quality of any ML model’s output depends entirely on the quality of its training data and the rigor of the training process. When something goes wrong in that pipeline, models can learn the wrong thing entirely—a phenomenon called shortcut learning.
A well-documented example involves chest X-ray analysis. Researchers found that AI models trained on open-source radiographic data for pneumothorax had learned to associate the presence of chest drainage tubes with the diagnosis rather than the underlying lung condition. The models performed well in testing because patients with pneumothorax often have chest tubes, but they systematically missed cases where the tube was absent. The AI had found a statistical shortcut rather than learning to identify the pathology itself.
Automation Bias and the Human-in-the-Loop
This failure mode points to a broader and more insidious risk: the tendency of humans to trust AI output uncritically. An ML model that confidently misdiagnoses because it has learned the wrong shortcut becomes a danger to patients when a clinician accepts its output without scrutiny.
Every deployment of ML in healthcare must therefore be designed on the assumption that the model will sometimes be wrong, and that the human in the loop is the last line of defense. This risk is sharpened by an awkward truth about how ML systems present their results. A language model is optimized to produce plausible-sounding prose, while a diagnostic classifier may return a confident probability score. A result that looks right is far more likely to be accepted without scrutiny. This tendency—automation bias—is perhaps the single greatest threat to the safe use of AI in medicine.
Edge AI: Intelligence at the Point of Care
A particularly promising branch of ML in healthcare is Edge AI, where models run directly on a local device rather than on remote servers. Unlike cloud-based systems, Edge AI processes data where it is captured, reducing dependence on an internet connection, lowering latency, and limiting the amount of sensitive information transmitted across a network.
An example is Neocam, a handheld screening tool invented by Dr. Louise Allen, a consultant pediatric ophthalmologist at Addenbrooke’s Hospital, Cambridge. It uses digital imaging to detect congenital cataracts in newborns. The device’s development was supported by training an ML model on 46,000 images to help midwives confirm that photographs are of sufficient quality for diagnosis. Neocam is currently being trialed across 30 UK maternity units in the NIHR-funded DIvO study, with over 140,000 babies expected to be screened.
Because processed summaries leave the device rather than raw sensor streams, Edge AI systems require dramatically less network bandwidth than cloud-dependent equivalents. This lighter communication load can extend battery life. Hybrid systems combine Edge AI for immediate on-device analysis with cloud models for deeper, longer-term insights, balancing responsiveness and privacy with greater processing power.
The Regulatory Landscape
Regulation must evolve alongside a technology whose risks do not stop changing once a device reaches the market. In September 2025, the MHRA launched the National Commission into the Regulation of AI in Healthcare. The Commission brings together clinical, technical, regulatory, and patient perspectives to address fundamental questions about how AI medical devices should be governed. As regulators in the UK and EU continue to refine their approaches, it will become clear whether these emerging frameworks will support innovation while maintaining patient safety.
A Note on ‘Intelligence’
Perhaps the most misleading part of artificial intelligence is the word ‘intelligence.’ These systems do not understand a diagnosis, recognize the human significance of a result, or know when the situation falls outside their experience. They identify patterns and calculate probabilities—sometimes with extraordinary speed and accuracy—but without comprehension.
That distinction matters, but it should not obscure their potential. A tool does not need to think like a clinician to become an integral part of clinical practice. Its value lies in supporting human expertise, not attempting to replace it. The opportunity is considerable, provided enthusiasm does not outrun scrutiny.
Machine learning will not shape healthcare on its own. Its impact will depend on how it is designed, governed, and used in practice. Handled thoughtfully, it could open a remarkable new chapter in medicine. What happens next depends not on how intelligent the machines become, but on how intelligently we choose to use them.
References
University College London, “Superhuman 20 second AI heart tool begins NHS roll-out,” March 2022. https://www.ucl.ac.uk/news/2022/mar/superhuman-20-second-ai-heart-tool-begins-nhs-roll-out
Banerjee et al., “‘Shortcuts’ Causing Bias in Radiology Artificial Intelligence: Causes, Evaluation, and Mitigation,” Journal of the American College of Radiology 20, 842–851 (2023). https://www.sciencedirect.com/science/article/abs/pii/S1546144023005264
Cambridge University Hospitals NHS Foundation Trust, “AI power improves cataract test tool for babies,” April 2025. https://www.cuh.nhs.uk/news/ai-power-improves-cataract-test-tool-for-babies/
MHRA, “National Commission into the Regulation of AI in Healthcare,” launched 26 September 2025. https://www.gov.uk/government/groups/national-commission-into-the-regulation-of-ai-in-healthcare
