The world is generating more biopsies, complex cases, and data than ever before, and the people best qualified to interpret them are in increasingly short supply. Something has to give – and that something is the move to digital pathology.
There's another strong driver supporting this move: digital pathology is the prerequisite for AI. You cannot feed a glass slide to a machine-learning model. Without digitization, the extraordinary advances in computational pathology of the last decade are simply inaccessible. Whole-slide image analysis, AI-powered tumor segmentation, predictive biomarker extraction from routine H&E stains: none of this becomes real for clinical practice until the field goes digital. The investment in digitization is therefore also an investment in everything that comes after it.
But, as with the implementation of any new technology, the benefits only bear fruit once the risks are fully appreciated and effectively mitigated. This article examines the ways in which AI-augmented diagnostics could be misleading, and some practical strategies for their risk management.
When the machine is wrong and you follow it anyway
Suppose your AI assistant flags a slide as low-grade. You have looked at the case for eight seconds, formed an impression, and, consciously or not, felt a small sense of relief that the system agrees. But what if your first instinct was right and the AI was wrong? What happens next is the phenomenon that should be keeping everyone in digital pathology up at night: automation bias.
Automation bias is not about blindly trusting computers. Most pathologists are far too experienced for that. Instead, it is a subtler failure: the tendency to allow automated output to anchor, nudge, and occasionally override what our own trained eyes are telling us.
In their landmark 2010 review, Parasuraman and Manzey defined two distinct error types that arise from this bias. Errors of omission occur when a clinician fails to detect a problem precisely because the automated system raised no alert, and vigilance quietly switches off. Errors of commission occur when a clinician acts on an incorrect automated recommendation without cross-checking it against other evidence. Neither form requires a credulous or careless clinician. Both can affect experts under everyday working conditions. The report states: “In 7 percent of cases, initially correct evaluations were overturned by erroneous AI advice, a figure that represents real patients receiving wrong answers.”
The clearest evidence for this risk in computational pathology comes from a 2024 study by Rosbach et al., titled “Automation Bias in AI-Assisted Medical Decision-Making under Time Pressure in Computational Pathology.” The team designed a web-based experiment in which 28 trained pathology experts estimated tumor cell percentages from whole-slide images, with and without AI assistance, and under varying degrees of time pressure.
The headline result was reassuring in one sense – overall, AI integration led to a statistically significant improvement in performance. But embedded within that improvement was a troubling signal: a 7 percent automation bias rate was measured. In 7 out of every 100 cases where a pathologist had initially reached the correct conclusion, the introduction of an erroneous AI recommendation caused them to abandon that correct answer. In a discipline where mistakes mean missed cancers or incorrect staging, 7 percent is not a rounding error; it is a patient safety number.
How might time pressure add to automation bias? The assumption might be that a stressed, rushed pathologist would simply defer to whatever the AI says and move on. The results of the Rosbach study were more nuanced. Time pressure did not significantly increase the rate at which automation bias occurred; pathologists were not flipping more frequently to the AI’s wrong answers when rushed. But it did appear to increase the severity of the bias: when pathologists did follow an erroneous recommendation under time pressure, they deviated further from the correct answer. They did not just make an error; they made a bigger one.
This pattern matters because time pressure is not a hypothetical scenario, but the daily reality of high-volume pathology departments. A system that performs well in a calm, unhurried experiment but degrades under real-world conditions is not the system we need.
And it is not only diagnostic accuracy that is at stake: a growing body of evidence now suggests that repeated reliance on AI decision support may progressively erode the manual skills and independent judgment of the clinicians using it. This "AI-induced deskilling" has been documented across radiology, colonoscopy, and computational pathology alike – a convergent finding that adds urgency to the question of how we design human-AI collaboration from the outset.
Five strategies to keep bias in check
Awareness of the problem is a necessary starting point, but it is not enough. Automation bias is partly structural, baked into how AI outputs are displayed, the workflows they sit within, and the institutional cultures around them. Avoiding it requires deliberate design at multiple levels. Here are five approaches that evidence and expert consensus are beginning to converge on.
Here are five approaches that can be derived from current evidence and established principles of human factors research.
1. Mandate explainability and make it meaningful
A number showing 87 percent probability of malignancy is not an explanation. For a pathologist to critically evaluate an AI recommendation, they need to see what the model actually found: which regions of the slide drove the prediction, what morphological features were weighted most heavily, and where uncertainty is high.
Explainable AI tools – such as attention maps, saliency overlays and confidence intervals – make the machine’s reasoning visible and, therefore, contestable. Importantly, explainability should be designed not merely to inform, but to prompt active evaluation. A recommendation accompanied by visible reasoning is harder to accept passively than a single probability score.
2. Enforce structured independence before AI disclosure
One of the simplest and most powerful workflow interventions is sequencing.
Based on findings from automation bias research and established principles of cognitive psychology, having pathologists form and record their own assessment before AI output is revealed is a plausible strategy to support cognitive independence.
This is not about distrust; it is about protecting cognitive independence. When a clinician has already committed to a conclusion, they engage with a subsequent AI recommendation more critically, assessing it against their own reasoning rather than using it as a starting point.
Some groups are beginning to explore workflow designs that delay the presentation of AI output until after an initial human assessment has been recorded, in order to reduce anchoring effects (Figure 1). Such workflows can be adapted to different levels of expertise and clinical contexts.
3. Design AI alerts that encourage verification, not acceptance
The interface through which AI results are presented is rarely neutral. Confident, prominent AI outputs, especially those presented without uncertainty ranges, create what researchers call a "complacency trap". The cognitive load of dismissing a clear system recommendation is high, so we tend not to.
Redesigning AI output interfaces to make uncertainty visible – to present recommendations as "for review" rather than "confirmed," and to include explicit prompts for verification in discordant cases – can materially reduce the rate of uncritical adoption. Design choices that feel small, including the wording of a label and the color of a confidence marker, turn out to matter considerably.
4. Build AI literacy into continuing professional development
Automation bias does not only affect the naive. But pathologists who have received specific training in how AI systems fail – including their systematic blind spots, training-set biases, and tendency to be confidently wrong on out-of-distribution cases – are better equipped to apply critical scrutiny.
CPD programs need to go beyond general AI ethics to include hands-on calibration exercises: cases where the AI is demonstrably wrong, used to train intuition for the situations where skepticism is most warranted. The risk of deskilling is real and can only be countered by the deliberate maintenance of independent diagnostic skills that AI is supposed to augment, not replace.
5. Establish institutional governance and feedback loops
Automation bias at the individual level will not be solved purely at the individual level. Hospitals and laboratories deploying AI in pathology need governance frameworks that include ongoing monitoring of AI performance, structured mechanisms for pathologists to flag discordant cases, and regular review of instances where human-AI disagreement occurred.
Who overturned whom, and was the human or the AI right? These questions, tracked systematically, reveal where bias is accumulating in real practice and allow both the AI system and the human workflows around it to be recalibrated. Without institutional accountability, automation bias remains invisible in the aggregate, even as its effects are felt one patient at a time.
A tool that demands active partnership
Digital pathology and AI hold genuine promise for a specialty under enormous pressure. The efficiency gains are real, as are the diagnostic augmentation, and the potential to extend high-quality diagnostic access to underserved settings. None of that disappears because automation bias exists.
What it means is that the transition to digital pathology cannot be treated as simply installing better software. It is a fundamental shift in how pathologists think, how laboratories are organized, and how we safeguard the critical human judgment that makes the difference between a tool and a crutch.
The glass slide may be going away, but the irreplaceable expertise of the person reading it is not. Protecting that expertise from its own new vulnerabilities is one of the most important tasks the profession faces in the decade ahead.
The goal is not for the pathologist to catch up with the machine. It is for the machine to augment the pathologist, not replace their thinking.
Sebastian Casu is Co-Founder and Chief Medical Officer of Elea.ai GmbH, a deep-tech company developing AI-powered software in pathology to optimize clinical workflows and documentation in healthcare. As a board-certified specialist in anesthesiology, intensive care, and emergency and palliative medicine, he previously served as Chief Physician and Medical Director in Hamburg. As a keynote speaker, he focuses on digital transformation, change management, and communication in healthcare.
Newsletters
Receive the latest pathologist news, personalities, education, and career development – weekly to your inbox.
