UK Asylum AI Summaries Fail 22% of Expert Checks: GPT-4o Stumbles in High-Risk Scenarios

Title

The UK Home Office's AI summarization tool deployed in asylum interviews was found by internal expert reviews to have roughly 22% of summaries failing the first round of review. This story was uncovered by the campaigning organization Open Rights Group (ORG) through six months of Freedom of Information requests and ministerial questioning, with The Register and the immigration-focused newsletter Thoughts on Immigration separately cross-verifying the core figures. For any team stuffing large models into high-risk workflows, this is a rare cautionary tale.

Two tools, one underlying model

According to ORG's investigation, the Home Office uses two generative AI tools in asylum casework. The first is Asylum Case Summarisation (ACS), which converts interview transcripts into summaries for caseworkers and has been available to all asylum caseworkers since April 2026. The second is Asylum Policy Search (APS), a conversational tool for retrieving government country policy documents, used by all asylum decision-makers since July 2025.

Both run on OpenAI's GPT-4o via Microsoft's Azure AI Foundry service. That means a general-purpose large model is embedded directly into an administrative process that determines individuals' fates, and ORG notes that applicants neither know their case was summarized by AI nor can view or correct those summaries.

Azure AI Foundry is Microsoft's enterprise-grade platform for developing and deploying AI applications, formerly known as Azure AI Studio. It aggregates models from OpenAI, Meta, Mistral, and others as managed APIs, bundled with content filtering, evaluation, monitoring, and compliance tools, making it easy for governments or large enterprises to integrate large models into internal systems. For buyers, its appeal is "one-stop compliance"—models, compute, logs, and access control all under the same cloud account, simplifying security reviews for government procurement. But this also means that once a general model is plugged into Foundry, the model's black-box nature is not eliminated; prompt management, output quality sampling, and audit responsibility still fall on the deployer. The Home Office called GPT-4o through this platform but did not simultaneously establish a matching quality-check mechanism.

What the inspection data reveals

The figures from The Register, ORG, and Thoughts on Immigration largely align, but the statistical scope is not entirely consistent. Of 203 summaries that underwent first-round expert review, 45 failed, about 22%; 19 were flagged for further review, and of the 56 that went to a second round, 30 still fell short, a failure rate of about 54%.

More alarming is the inspection coverage: between April and July 2026, experts sampled only about 0.3% of AI summaries. At such a low sampling rate, a systemic problem can run for months undetected. ORG also disclosed that during the 2025 pilot phase, 9% of summaries were "unfit for use," and in subsequent larger-scale sampling tests, that proportion rose to 15%—quality deteriorated as scale increased.

According to The Register, the Home Office confirmed in August that expert testing of APS had stopped, though it had previously told MPs that checks were continuing. ORG says the Home Office refused to share how staff were trained, the prompts used by the tools, or the quality-testing methodology, and some responses contradicted each other. Neither tool is listed in the government's public register of algorithmic tools.

The Register also noted that since April, the average monthly number of initial asylum decisions has dropped by about 42% compared with the same period in 2025, and the time from interview to decision increased by 29%. The outlet explicitly states that this does not prove the AI tools caused either change.

Statistically, a 0.3% sampling rate means roughly 1 in every 333 summaries was seen by an expert. If a type of error occurs at 22% in the population, drawing 203 samples gives a high probability that the observed failure rate is near the true value, but it does not guarantee that low-probability, high-harm errors (such as summarizing a key date or location incorrectly) are caught. To keep error within an acceptable range, sample sizes typically need to reach hundreds to thousands and cover different languages, nationalities, and interviewers. More critically, a low sampling rate produces wide confidence intervals, making it hard for the Home Office to claim "systemic quality is good." In a regulatory context, inspection coverage is itself an auditable metric, not merely a quality metric.

Five things developers and buyers should remember

The value of this incident is not in mocking a government department's procurement decision, but in exposing the common flaws of AI summarization systems in high-risk scenarios.

Sampling rates cannot be so low as to be meaningless. A 0.3% inspection ratio means the vast majority of outputs were never seen by a human. Any AI system deployed in a real decision chain needs human review coverage set at a meaningful minimum threshold, not a symbolic few samples after launch.

Pilot data cannot substitute for evaluation under real load. The pilot's 9% failure rate became 15% or higher at scale. Pilot environments usually have cleaner inputs and more cautious users; once in daily high-pressure workflows, the text quality and edge cases the model faces all worsen.

Give the people being summarized the right to view and correct. Asylum applicants do not know AI summarized their interview and cannot correct errors. A basic design principle: when a system summarizes what someone said, that person should have the opportunity to review and correct it.

Keep auditable records. ORG could not obtain prompts or test results, making external audit nearly impossible. Recording prompt versions, model versions, and evaluation results is not just a compliance requirement—it is the only basis for self-justification or self-correction when the system is challenged.

Beware of automation eating the existing review layer. ORG warns that AI summarization may remove a layer of review previously provided by human experts. Automating a step can easily remove the quality assurance that step once provided.

"Automation eating the review layer" is known in safety engineering as "automation bias" and "responsibility diffusion." When a system provides seemingly authoritative summaries, human reviewers tend to lower their guard, treating the summary as fact rather than a draft to be verified, so errors that manual checking would have caught are missed. More insidiously, organizations often cut manpower after introducing automation, making the cost of returning to manual processes extremely high, creating path dependency. For high-risk AI systems, feasible countermeasures include: explicitly requiring human reviewers to check certain key fields against the original record, labeling summaries as "draft" in the interface, periodically running "human-machine comparison" experiments to measure reviewers' actual catch rates, and keeping a full manual fallback plan ready to restore at any time.

Controversy and demands

ORG calls for pausing both tools, conducting an independent audit of their accuracy in real-world use and publishing the results, and for the Independent Chief Inspector of Borders and Immigration to intervene. ORG project manager Sara Alsherif put it more aggressively: "We urge the government to regulate AI and completely ban its use in asylum applications."

The Home Office did not respond to The Register's questions. The core disagreement now is: the Home Office seems to believe the tools operate within controlled limits, while ORG's records show extremely low inspection coverage, testing stopped, and FOI requests evaded. Until an independent audit is conducted, the dispute is unlikely to be resolved. But the two figures—22% first-round failure rate and 0.3% sampling rate—placed together are enough to make the point.

One-line Summary

UK Home Office's GPT-4o asylum summary tool failed 22% of first-round checks, with only 0.3% sampled.

Paragraph Summary

The UK Home Office's two GPT-4o-based AI tools deployed in asylum interviews have been exposed for quality problems. Data unearthed by Open Rights Group through FOI requests and cross-verified by The Register and others show that of 203 expert-sampled ACS summaries, 45 failed the first round, about 22%, with a second-round failure rate of 54%; and between April and July 2026 experts checked only about 0.3% of summaries. The pilot-phase 9% failure rate rose to 15% at scale. Applicants are unaware and cannot view or correct summaries; the Home Office refuses to disclose prompts and testing methods, and the tools are not on the algorithmic register. This serves as a cautionary tale for developers and buyers of high-risk AI workflows: sampling rates must not be meaningless, pilot data cannot replace real-load evaluation, those summarized should have the right to view and correct, auditable records must be kept, and beware of automation removing the human review layer.