NederlandsDiagnostic AI AMIE tested with real patients in a GP practice for the first time
Google's medical AI AMIE held conversations with 100 real patients at a US GP practice before their appointment with the doctor. Not a single conversation had to be stopped for safety reasons. The study is in The Lancet; larger trials are the next step.
Google has tested its medical AI system AMIE with real patients in an ordinary GP practice for the first time. AMIE is a chatbot that, like a doctor at a first consultation, asks questions about symptoms and lists possible causes. The study, carried out with Beth Israel Deaconess Medical Center (BIDMC) in Boston, appeared in early October in the medical journal The Lancet. According to Google, it is the company's first publication in the main journal of that title. An earlier version of the study had been online since March.
How the trial went
Between April and November 2025, 100 adult patients chatted with AMIE, between zero and five days before they came to an urgent appointment at the BIDMC GP practice with a new, non-life-threatening complaint. Of these, 98 actually came to that appointment. AMIE ran on Google's language model Gemini 2.5, without extra medical training.
During every conversation, a doctor watched live over a video link. The doctor could stop the conversation in four situations agreed beforehand, such as danger to the patient, great distress or a request from the patient. That was not needed in any conversation. Afterwards, with the patient's consent, the treating doctor received a record of the conversation and a summary, before the consultation.
What came out of it
A group of doctors, who did not know who had written what, assessed the lists of possible diagnoses and the proposed treatment plans from AMIE and from the GPs. They found no clear difference in the quality of the diagnoses or in the safety of the plans. The GPs did score better on how practical and how affordable their plans were. That is to be expected, the researchers write: AMIE could not see the patient's records, could not examine the patient and could not see the patient.
Eight weeks later, the researchers looked in the records to see what the final diagnosis had turned out to be. In 90 per cent of cases it was on AMIE's list of possible diagnoses. In 75 per cent it was among the top three, and in 56 per cent it was AMIE's first choice.
Of the GPs who filled in a questionnaire, 75 per cent found AMIE's record useful for preparing. More than half said it had probably or certainly changed their approach. According to the researchers, the consultation as a result was less about collecting information and more about checking it. The patients were also satisfied, and after the conversation with AMIE they thought clearly more positively about AI than before.
What still has to happen
It was a small trial at one practice, without a comparison group of patients who did not have a conversation with AMIE. The system worked with text only, and the participants were younger than the average visitor to the urgent appointments: there, more than half were over 60. That is why the researchers now want larger studies with such a comparison group, before AI starts talking to patients on a large scale.
What does this mean for the road to the singularity?
Until now, medical AI has mostly been tested with actors playing a patient. This trial shows that such a system can also talk safely with real patients in a real practice, and usually has the right diagnosis in view. If larger studies confirm this, AI could prepare the first conversation with the patient, so that the doctor has more time left for examination and explanation. The next step is an AI that can also read the records and perceive more than text.
What is the technological singularity? Read the explanation →
