LONDON — AI scribes used in NHS consultations can wrongly record medicines and diagnoses, creating a risk to patient safety and medical records, Healthwatch England has warned.
The statutory patient champion said it had seen cases where patients, rather than clinicians, spotted the mistakes. In one example, a woman was told an AI summary said she had demyelination, a serious nerve condition, when the intended result was “null demyelination”.
In another case, an AI tool mixed up the name of a prescribed drug with a different medicine with a similar name. Healthwatch said such errors could persist if they are not challenged and corrected.
Patients say AI summaries can leave serious errors in records
Healthwatch England said one patient was badly shaken after an AI-generated note appeared to give her an incorrect diagnosis. The woman, who works in healthcare and did not want to be named, said she only got the record corrected after querying the MRI result herself.
She said the experience was traumatising, particularly because the wrong term suggested a neurological problem that can be associated with multiple sclerosis. The hospital later corrected the record.
Healthwatch also described a case in which an AI-generated letter omitted advice from a hospital consultant that a patient should get a repeat migraine prescription from their GP. That kind of omission could leave a patient unable to obtain medication on time.
The watchdog said it had heard multiple accounts of patients catching errors that staff had not noticed, raising concern about how often mistakes may go undetected in routine use.
Rapid rollout of AI scribes outpaces safety checks, watchdog says
Healthwatch said the technology is being rolled out quickly as part of the NHS’s wider push to move from analogue systems towards digital ones. The government’s 10-year health plan for England expects AI scribes to reduce bureaucracy and give staff more time with patients.
But the organisation warned that there is still no clear route for patients to report mistakes and get them corrected, whether the error came from the tool itself or from the professional using it. A Healthwatch spokesperson said healthcare has never been error-free, but the findings showed an urgent need for clarity.
The watchdog also said it was worrying that the Medicines and Healthcare products Regulatory Agency has decided not to classify AI scribes as medical devices. That means there will be no England-wide oversight to check they are safe and effective.
Doctors and researchers say the tools can still hallucinate and misread consultations
GPs and hospital doctors in England are already using 27 different AI scribes. Ministers have been warned that the NHS and individual clinicians could face legal action if mistakes made by the systems harm patients.
Doctors have also raised concerns that the tools may not save time if every transcript has to be checked line by line. Dr Shier Ziser Dawood, a London GP, wrote last year that AI scribes could be a “double-edged sword” and described one case where a system claimed she had told a patient to continue Prozac when she had not discussed the drug at all.
That kind of error is known as a hallucination, when AI refers to something that was never said in the consultation. Dr Charlotte Blease, an AI expert at Uppsala University, said mistakes are especially likely in consultations with more than one person, complex medical histories, or patients whose first language is not English.
She said doctors can make errors too, but it is possible the AI error rate is worse. In her survey of 1,003 UK GPs, more than half said ambient AI records were more accurate than notes they produced themselves.
The latest UK news and stories from MCFC Report.
