ARISE, BIDMC and Google AMIE publish prospective clinical AI feasibility study in The Lancet
ARISE Network Founding Site Lead Adam Rodman and Google's AMIE team have taken conversational medical AI beyond simulated consultations and into a study with real patients. Published in The Lancet, the prospective clinical feasibility study at Beth Israel Deaconess Medical Center (BIDMC) examines how AI can help conduct clinical history taking with patients under physician supervision and enhance subsequent patient-physician encounters.
The collaboration combines both teams' research and technical expertise to evaluate healthcare AI in an ambulatory primary care workflow. It is a shared research milestone for ARISE and Google teams, adding prospective evidence about patient experience, clinician use and safety oversight in a real clinical setting.
From simulated consultations to real patients
In the study, 100 adults completed a text conversation with AMIE before an urgent care appointment at BIDMC's primary care practice. AMIE gathered a clinical history and discussed possible diagnoses for patients to raise with their clinician. The clinician could review the conversation and a summary before the visit. A physician monitored every AI conversation live, with predefined safety criteria for stopping an interaction.
No conversations required a safety stop under those criteria. Patients reported high satisfaction, and clinicians who reviewed AMIE's outputs described benefits for visit preparedness. These are encouraging safety findings from a closely supervised research workflow, not evidence that unsupervised AI is ready to be used in clinical care.
Among the 98 participants who completed both the AI conversation and the clinical visit, AMIE's list of seven possible diagnoses included the diagnosis established through subsequent chart review in 90% of cases; its top three included that diagnosis in 75%.
Blinded reviewers did not find statistically significant differences in overall differential-diagnosis quality or management-plan appropriateness and safety. Clinicians' management plans scored better for practicality and cost effectiveness.
A collaborative milestone
Earlier AI studies relied primarily on synthetic vignettes or simulated consultations with trained patient actors. This prospective feasibility study shows how AI systems might fit into actual clinical workflows, providing evidence about patient experience and clinician use that informs future potential adoption and implementation.
See also the team's recent Nature Medicine commentary “Prospective evidence for conversational medical AI is hard, but non-negotiable”.
Acknowledgements
We acknowledge the full author team: Peter Brodeur; Jacob M. Koshy; Anil Palepu; Khaled Saab; Ava Homiar; Roma Ruparel; Charles Wu; Ryutaro Tanno; Joseph Xu; Amy Wang; David Stutz; Wei-Hung Weng; Hannah M. Ferrera; David Barrett; Lindsey Crowley; Jihyeon Lee; Spencer E. Rittner; Ellery Wulczyn; Selena K. Zhang; Elahe Vedadi; Christine G. Kohn; Kavita Kulkarni; Vinay Kadiyala; S. Sara Mahdavi; Wendy Du; Jessica M. Williams; David Feinbloom; Renee Wong; Tao Tu; Petar Sirkovic; Alessio Orlandi; Christopher Semturs; Yun Liu; Juraj Gottweis; Dale R. Webster; Joëlle Barral; Katherine Chou; Pushmeet Kohli; Avinatan Hassidim; Yossi Matias; James Manyika; Rob Fields; Jonathan X. Li; Marc L. Cohen; Vivek Natarajan; Mike Schaekermann; Alan Karthikesalingam; Adam Rodman. We also acknowledge the authors of the Nature Medicine commentary: Mike Schaekermann; Anil Palepu; Adam Rodman; Ami Parekh; Ethan Goh; Shekoofeh Azizi; Yun Liu; Sunny Virmani; Christina Chen; Dale R. Webster; Joëlle Barral; Avinatan Hassidim; Yossi Matias; Po-Hsuan Cameron Chen.
