An artificial intelligence (AI) system that selects diagnostic tests and progressively refines its conclusions achieved greater diagnostic accuracy than generalist physicians in simulated clinical encounters, according to research published in Science.
Developed by Microsoft researchers, the system, called Diagnostic Orchestrator (DxO), uses large language models to simulate a panel of clinical experts. Rather than interpreting a complete patient record, it begins with limited information, asks questions, requests investigations, and updates its diagnosis as new evidence becomes available.
In the study, researchers developed a testing framework, SDBench, comprising 304 clinicopathological conference cases published in the New England Journal of Medicine and 325 emergency department encounters. The framework assessed both diagnostic accuracy and the estimated costs of investigations.
Using GPT-5, DxO achieved 83 percent accuracy on the clinicopathological cases, compared with 36 percent among physicians with access to reference materials. For emergency department cases, accuracy was 80 percent versus 47 percent.
Importantly, the system demonstrated an ability to balance diagnostic performance against testing expenditure. In the clinicopathological cases, DxO achieved 81 percent accuracy at an average estimated testing cost of $2,241, compared with $4,725 for physicians achieving 36 percent accuracy.
The findings suggest opportunities for AI-assisted diagnostic test selection. DxO generally asked more questions than clinicians before ordering investigations, potentially identifying useful clinical clues that could reduce reliance on extensive testing panels. However, while the model can request examination findings, it cannot perform physical exams or observe patients directly.
The researchers emphasize that these results were obtained under controlled, text-based conditions. The evaluation excluded important elements of routine practice, including pathology slides, imaging data, incomplete clinical records, and multidisciplinary decision-making. Testing costs were also estimates rather than observed expenditure.
Further evaluation will be required to establish the model's effectiveness in real-world clinical practice and its impact on patient outcomes. Nevertheless, the study suggests that AI could contribute to diagnostic stewardship by helping clinicians determine not only which tests to request, but also when sufficient evidence has been gathered to reach a diagnosis.
Newsletters
Receive the latest pathologist news, personalities, education, and career development – weekly to your inbox.
