Clinical Scorecard: Why Accurate Pathology AI Is Not Enough

At a Glance

Key Highlights

• High diagnostic accuracy alone does not establish clinical value or demonstrate improved patient outcomes.
• Tissue fixation, staining, section thickness, scanners, and laboratory procedures can affect image quality and AI performance.
• The authors recommend external validation in representative patient populations, local testing, and ongoing performance monitoring.
• In research involving 28 pathology experts, incorrect AI recommendations overturned previously correct assessments in approximately 7% of consultations.
• The proposed six-stage framework is conceptual and has not been validated in prospective studies.

Guideline-Based Recommendations

Diagnosis

• Evaluate whether AI supports appropriate diagnostic decisions, not only whether it achieves high technical accuracy.

Management

• Assess compatibility with existing laboratory systems and workflows before adoption.
• Test performance across different scanners and patient populations.
• Evaluate how pathologists respond to AI-generated recommendations.

Monitoring & Follow-up

• Monitor AI performance after implementation.
• Reassess performance when laboratory procedures, software, or patient populations change.

Risks

• Differences in tissue preparation, image acquisition, scanners, and laboratory procedures may affect AI performance across settings.
• Inappropriate reliance on AI recommendations may introduce diagnostic errors.

Patient & Prescribing Data

The editorial discusses representative patient populations as necessary for external validation; it does not specify a particular patient group.

Not applicable; the article addresses evaluation and implementation of pathology AI rather than patient treatment.

Clinical Best Practices

• Use external validation with representative patient populations and conduct local testing.
• Assess the full diagnostic pathway, from tissue preparation and image acquisition through workflow, clinical decisions, and patient outcomes.
• Seek evidence that AI integrates safely into laboratory workflows and supports appropriate diagnostic decisions.
• Continue performance monitoring after implementation, especially when relevant laboratory conditions change.

Related Resources & Content

• Journal of Pathology Informatics editorial, “Why Accurate Pathology AI Is Not Enough”