Washington study: Speech AI understood the accents perfectly, but still gave different scores by accent

New research from the University of Washington shows that leading audio models from Google, OpenAI and Alibaba understand accented English perfectly – yet still rate speakers differently based on accent.

Illustration: microphones with red thread sound waves, one thread tangled — AI that understands accents but judges them unequally.
Illustration

Washington study: Speech AI understood the accents perfectly, but still gave different scores by accent

New research from the University of Washington shows that leading audio models from Google, OpenAI and Alibaba understand accented English perfectly – yet still rate speakers differently based on accent. That makes the finding strong evidence that the problem is not comprehension, but a hidden hierarchy in the evaluation itself.

The Core Finding: Comprehension Was Correct, the Scores Were Not

The University of Washington study examined how some of the most advanced audio models score English speakers with different accents, both on what they said and how they said it. In the scripted tests, every speaker read exactly the same words, and the models transcribed the accented speech correctly – meaning the models understood the accents perfectly well. Yet the models still gave the speakers different scores depending on their accent.

This combination makes the finding stronger than the opposite scenario would have been. If the models had simply misunderstood Chinese or Nigerian English, the problem could have been framed as a technical transcription issue. Here, the content was identical and the transcription correct. The only thing left to explain the score differences was how the speech sounded – the accent itself.

The ranking was clear: speakers with American and British accents ranked at the top, speakers with Indian accents in the middle, and speakers with Chinese and Nigerian accents at the bottom (The Seattle Times).

The Method: Five Accents, Identical Scripts, a 1–7 Scale

For part of the test, the researchers used AI-generated "synthetic voices" that read identical scripts with the five different accents. The scripts mimicked three realistic situations: answers in job interviews, presentations at conferences, and answers on English tests. The models were asked to evaluate the recordings on a scale from 1 to 7, on both delivery (how something was said) and content (what was said).

Because the words were identical and the voices synthetic, the setup isolates accent as the variable: any score difference between the accents cannot be due to one speaker actually saying something better or worse than another.

The results also repeated when the synthetic voices were swapped out for human speakers. Both delivery and content were scored, and the accent pattern held on both.

Four Models, the Same Pattern – Including Alibaba's Own

The researchers tested Google's Gemini 2.5 Pro and Gemini 2.5 Flash, OpenAI's GPT-Audio-1.5, and Qwen3-Omni from the Chinese tech giant Alibaba. The models were among the most advanced available when the team ran the tests in spring 2026. Newer versions have since been released – so the findings cannot be straightforwardly transferred to today's models.

One of the most telling details: even Qwen3-Omni – the model from Alibaba, a Chinese company – gave British speakers the best delivery scores and Chinese speakers the worst, when the model listened to human interview answers. The pattern therefore does not appear to follow the companies' country of origin, but rather something in the training data or evaluation logic that is common across providers.

The models' own written justifications bore out the picture. They described American and British speakers as "clear" and "excellent," while Chinese and Nigerian speakers were described as "mumbling" or "difficult to understand" – despite the transcription showing that the model had understood them.

The Model Admits It Itself – Rarely

A separate part of the study examined whether the models could report their own bias. They could only do so to a limited extent: the models admitted that accent had influenced the scores in only about 10 to 30 percent of their justifications. In practice, that means if you ask an audio model why it gave one speaker a low score, it will in most cases offer a different explanation than the accent – thereby hiding the actual mechanism.

This makes the models' own explanations unreliable as a basis for decisions, and it makes the bias harder to uncover in operation without systematic tests of the kind the researchers conducted.

A Warning Against the "Judge Role" in High-Stakes Contexts

Study co-author Orevaoghene Ahia, a doctoral student at the University of Washington's Paul G. Allen School of Computer Science and Engineering, pointed out that these types of models are already being used as evaluators in situations with major consequences.

"People are now relying on audio AI models as a kind of third-party evaluator – for hiring, for oral exams in education, for certain English tests in immigration contexts – contexts where fairness really matters," Ahia said, according to The Seattle Times.

Ahia's own characterization of what happens inside the models was not that they are openly biased, but something more subtle: "They understood the speakers, but they just had this implicit notion of a hierarchy between them based on accent," he said, according to South China Morning Post.

That is an important nuance: the study does not document that the models were designed to rank accents, or that the bias is intentional. It documents a pattern in the results – a hierarchy the models themselves do not report, but which is clearly visible in the scores.

The study was presented in October at EMNLP 2026 (Conference on Empirical Methods in Natural Language Processing), and the coverage was published in syndicated form on November 20, 2026.

Caveats: Model Versions, Gender and Funding

Several caveats are worth keeping in mind. The tests were run in spring 2026, and the providers have since released newer model versions – it is unknown whether the bias remains the same in today's versions.

Gender, on the other hand, was not an explanatory factor: the results did not differ meaningfully between male and female voices, according to Ahia. Accent alone appears to be the driving variable.

The study was partially funded by gifts from Google and OpenAI – companies whose models were tested. Ahia nonetheless said the companies played no role in the work. A somewhat ironic detail in the media coverage: South China Morning Post, which republished the findings, is owned by Alibaba – the company behind Qwen3-Omni, which received critical results in the study. That does not change the findings themselves, which originate from Tribune News Service and the University of Washington, but it is worth noting for readers weighing the source picture.

Open Questions

The study leaves several questions open. Where does the implicit hierarchy come from – training data, evaluation criteria, or something else? Why do the results vary somewhat between providers, even though the main pattern is shared? And does the bias also hold in the versions of the models in use today?

The research group plans to continue its work by testing more accents, and to examine whether the pattern also applies to audio from emergency communications – an area where misjudging a speaker can have direct consequences for who gets prioritized.

The research group's practical conclusion is clear: as long as the bias is documented, and the models themselves are unable to admit it, audio AI should not serve as a judge in contexts where lives, careers or residence permits are at stake – at least not without protective measures and human oversight.

The Main Points

  • Leading audio models (Gemini 2.5 Pro/Flash, GPT-Audio-1.5, Qwen3-Omni) transcribed accented speech correctly, yet still gave different scores based on accent.
  • The ranking: American and British at the top, Indian in the middle, Chinese and Nigerian at the bottom – on identical scripts and on a 1–7 scale for delivery and content.
  • The models described American/British speakers as "clear" and "excellent," and Chinese/Nigerian speakers as "mumbling" or "difficult to understand."
  • The models admitted accent influence in only about 10–30 percent of their own justifications.
  • The study was presented at EMNLP 2026 in October; the tests were run in spring 2026 on model versions that have since been replaced by newer ones.
AIMag.no
AIMag.no
The AIMag.no editorial team covers artificial intelligence, tools, research, and regulation.

Get the best of AI MAG in your inbox

News, analysis, and ideas at the intersection of AI and society.