Human-AI Collaboration Enhances Clinical Reasoning in Ophthalmology but Risks Overconfidence and Automation Bias

The New Frontier of Clinical Reasoning: Bridging the Gap with Human-AI Collaboration

Clinical reasoning is the cornerstone of medicine, yet it remains one of the most complex tasks for clinicians to master, particularly in highly specialized fields like ophthalmology. As large language models (LLMs) continue to evolve, the concept of Human-AI Collaboration (HAC) has moved from theoretical discourse to experimental reality. A recent study by Ong et al., published in the International Journal of Medical Informatics, provides a critical evaluation of how conversational AI, specifically Claude-3.5-Sonnet, assists medical professionals in navigating challenging real-world cases.

While previous research has focused on the standalone performance of AI models, this study shifts the lens to the interaction between the machine and the clinician. The findings offer a nuanced perspective: AI can be a powerful diagnostic ally, but its integration into the clinical workflow is fraught with psychological and methodological hurdles that must be addressed to ensure patient safety and diagnostic precision.

Highlights

• HAC significantly improved mean diagnostic accuracy from 0.45 to 0.60 across a cohort of students, residents, and board-certified ophthalmologists.
• The accuracy of the AI working alone (0.70) exceeded the accuracy of the Human-AI collaborative effort (0.60), highlighting a ‘collaboration gap.’
• Collaboration significantly increased clinician confidence and reduced cognitive burden, even in instances where the final diagnosis was incorrect.
• The success of the collaboration was highly dependent on the baseline difficulty of the case, with significant gains seen only when human accuracy was above a certain threshold.

Background: The Challenge of Specialized Diagnostic Reasoning

Diagnostic errors remain a significant concern in healthcare, often stemming from cognitive biases, information overload, or the sheer complexity of rare clinical presentations. In ophthalmology, where diagnosis often relies on subtle visual cues and integrated systemic knowledge, the stakes are high. LLMs have demonstrated remarkable capabilities in passing board exams and providing differential diagnoses, but their role as a ‘co-pilot’ in real-time reasoning is less understood. The central question of the Ong et al. study was whether a conversational interface could truly augment human intelligence or if it would merely introduce new forms of bias, such as automation bias—the tendency to over-rely on automated systems.

Study Design: A Rigorous Crossover Experiment

The researchers employed a crossover experimental design to minimize individual variability. The study population consisted of 30 participants divided into three groups: 10 board-certified ophthalmologists, 10 ophthalmology residents, and 10 senior medical students. This stratification allowed for an assessment of how clinical experience influences the effectiveness of AI collaboration.

The task involved solving 30 challenging cases sourced from JAMA Ophthalmology, known for their diagnostic complexity. Each participant completed cases under two distinct conditions:

1. Independent Work (Human-only): Participants reached a diagnosis using only their existing knowledge and the provided case materials.
2. Collaboration (HAC): Participants engaged in a free-text conversation with Claude-3.5-Sonnet to arrive at a diagnosis.

The primary endpoint was diagnostic accuracy. Secondary endpoints included self-rated confidence (measured on a Likert scale) and cognitive burden (assessed via the NASA Task Load Index). Furthermore, the researchers performed a deep dive into the interaction logs, categorizing the LLM’s behaviors into six patterns of accepting or arguing with the human user.

Key Findings: Significant Gains with a Persistent Gap

The Performance Paradox: HAC vs. AI-Only

The most striking result was the overall improvement in accuracy. The mean accuracy rose from 0.45 in the human-only condition to 0.60 in the HAC condition (P < 0.001). However, this improvement did not reach the level of the LLM-only performance, which stood at 0.70. This suggests that humans often 'filter out' or ignore correct insights provided by the AI, or conversely, the AI fails to convince the human when the human is set on an incorrect path.

Interestingly, the benefit of AI was not uniform. While 80% of participants saw an improvement or stability in their performance, 20% actually performed worse when collaborating with the AI. This decline often occurred when the AI provided plausible but incorrect information that the clinician then adopted, a classic example of automation bias.

Confidence and Cognitive Load: The Psychological Shift

One of the more concerning findings from a safety perspective was the impact on clinician psychology. HAC significantly increased self-rated confidence and reduced cognitive burden (P < 0.001 for both). While reducing burnout and increasing confidence are generally positive, these effects were observed even in 'failed HAC' sessions. In other words, the AI made the clinicians feel more certain and less stressed about their decisions, even when those decisions were wrong. This 'false sense of security' could lead to a reduction in the critical skepticism necessary for high-stakes medical decision-making.

When HAC Fails: A Behavioral Analysis

The researchers categorized the interaction patterns to understand why some collaborations succeeded while others failed. In successful HAC sessions, the most common pattern (92.6%) was the LLM presenting a correct insight which the human then accepted. In contrast, 58.6% of failed sessions involved the LLM presenting an incorrect insight that the human accepted without sufficient challenge. This highlights a critical vulnerability: clinicians may lack the ‘AI literacy’ or the specific subject-matter depth required to verify the AI’s suggestions when the case is at the edge of their expertise.

Expert Commentary: Navigating the ‘Uncanny Valley’ of Clinical AI

The study’s use of sliding paired t-tests revealed a vital ‘difficulty threshold.’ HAC was most effective when the human-only correct response rate was above 47%. When the cases were so difficult that human accuracy fell below 30%, the AI collaboration failed to provide a significant boost. This suggests that for the most ‘undiagnosable’ cases, current AI models might not yet provide the breakthrough required, or the human-AI interface is not yet optimized for extreme uncertainty.

From a clinical perspective, these results suggest that AI should be viewed as a ‘reasoning partner’ rather than an oracle. The fact that AI alone outperformed the human-AI team is a call to action for better interface design. Future systems must not only provide the correct answer but also provide the underlying evidence in a way that allows the clinician to critically appraise the logic. The goal is ‘augmented intelligence,’ where the final decision is superior to what either the human or the AI could achieve alone.

Limitations of the study include its focus on a single medical specialty and the use of a specific LLM (Claude-3.5-Sonnet). Different models or different clinical fields might yield varying results. Additionally, the experimental setting may not fully capture the time pressures and environmental distractions of a real-world clinic.

Conclusion: Implications for the Future of Medical Practice

The study by Ong et al. demonstrates that Human-AI Collaboration is a potent tool for enhancing diagnostic accuracy in complex ophthalmological cases. However, it also serves as a cautionary tale regarding the psychological impacts of AI. The reduction in cognitive burden and the boost in confidence must be balanced with rigorous clinical validation.

For medical educators, these findings suggest a need to incorporate ‘AI interaction skills’ into the curriculum. Clinicians must be taught how to argue with an AI, how to spot hallucinations, and how to maintain healthy skepticism. For health policy experts, the ‘collaboration gap’—where the team performs worse than the AI alone—indicates that we are still in the early stages of optimizing the human-machine interface. As we move toward a future where AI is ubiquitous in the clinic, the focus must remain on ensuring that these tools serve to sharpen, rather than dull, the clinical mind.

References

1. Ong KT, Seo J, Kim H, Kim J, Kim J, Kim S, Yeo J, Choi EY. Success and failure of human-AI collaboration in clinical reasoning: An experimental study on challenging real-world cases. Int J Med Inform. 2026 Feb 10;211:106342. doi: 10.1016/j.ijmedinf.2026.106342.
2. JAMA Ophthalmology. Case Records of the Massachusetts Eye and Ear Infirmary. (Source material for study cases).
3. Parasuraman R, Manzey DH. Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors. 2010;52(3):381-410.