Jan 11, 202621Research

Anthropic uses MedAgentBench from Stanford ARISE Network to evaluate Claude on medical agent tasks

Anthropic uses MedAgentBench from Stanford ARISE Network to evaluate Claude on medical agent tasks

Anthropic used Stanford’s MedAgentBench to compare Claude models on clinical task completion in an electronic health record (EHR) in its January 11, 2026 healthcare and life sciences announcement. Its chart reported MedAgentBench results alongside medical calculation accuracy on MedCalc, including evaluations of Opus 4.5.

MedAgentBench tests whether an agent can retrieve information and take actions to complete clinical work in a virtual EHR. Anthropic’s chart labels it as a Stanford benchmark and shows how performance changed across Claude models. The 4.5 results used extended thinking and native tool use. Anthropic used the public benchmark independently; clinical deployment requires separate validation.

Anthropic’s January 2026 healthcare announcement used MedAgentBench (green line) to compare medical agent task completion across Claude models. The figure labels MedAgentBench as a Stanford benchmark; the 4.5 results used extended thinking and native tool use. Source: https://www.anthropic.com/news/healthcare-life-sciences

The current MAST, or Medical AI Superintelligence Test, suite includes MedAgentBench v2 in its Agentic dimension. MAST is the ARISE Network’s independent medical AI evaluation platform, bringing clinical benchmarks together to compare models across medical capabilities and updating those comparisons as evaluations are added. Anthropic’s January evaluation used a different configuration from current MAST testing.

The original MedAgentBench is open source, and the v2 implementation is publicly available. Researchers can find MAST’s evaluation resources and model submission tools in its GitHub repository, with results and methodology and the model and benchmark submission process on the platform.

MedAgentBench was developed by Yixing Jiang, Kameron Black, Gloria Geng, Danny Park, James Zou, Andrew Y. Ng and Jonathan H. Chen.

References

Anthropic healthcare announcement, Jan 11, 2026: https://www.anthropic.com/news/healthcare-life-sciences

MedAgentBench paper and author list: https://arxiv.org/abs/2501.14654

MAST benchmarks and methodology: https://www.arise-ai.org/mast/benchmarks; https://www.arise-ai.org/mast/methodology

MAST evaluation resources: https://github.com/ARISENetwork/mast