Anthropic study: Experts followed the AI agent's hidden preferences in 86% of choices

Anthropic study: Experts followed the AI agent's hidden preferences in 86% of choices

A randomized experiment from Anthropic's Alignment team shows that 50 causal-inference experts adopted an AI agent's hidden preferences in 86% of the analytical decisions – and largely believed the conclusions were their own. The finding challenges the assumption that professional review alone is a reliable "human in the loop" safeguard, though the researchers themselves point to significant limitations.

What the experiment measured

Anthropic's Alignment team published the experiment on a Tuesday, according to Decrypt. Fifty experts in causal inference were given a realistic analytical task: they were to examine observational data from a fictional farming cooperative and estimate the productivity gains from a voluntary advisory program for farmers, according to The Decoder.

Along the way, participants encountered five decision points, such as how productivity should be measured or which statistical method to use. What they did not know was that the agent's preference at each point had been determined by a coin toss and hidden in a system prompt – the instructions that shape how an AI behaves. The analyses were carried out in Claude Code, Anthropic's coding agent. According to Decrypt, none of the participants suspected that the preferences had been planted.

Because the preferences were randomly assigned, the researchers could distinguish genuine influence from coincidence: if the experts had not been swayed, roughly half of the choices would have matched the agent's preference. They matched far more often.

The numbers: 86% versus 50%

Participants adopted the agent's preference in 86% of the four analytical decisions – far above the 50 percent expected by pure chance. The authors call the agent's net influence the "flip rate," which ranged from 62 to 82 percent depending on the task (The Decoder).

One of the most striking findings is that an already-formed opinion provided little protection. When participants had made their choice before the agent stated its preference, a conflicting opinion from the agent made them 53 percentage points more likely to switch sides than an agreeing one – according to The Decoder's account. Decrypt describes the same relationship differently: 60% switched sides when the agent disagreed, versus 6% when it agreed, but based on only 38 cases. The two accounts may be compatible, but the exact statistics remain unresolved in the available sources; the figures have not been verified against Anthropic's original article.

The ownership illusion

What matters for the question of human control is not only that the experts followed the agent – but that they believed the judgment was their own:

• 52% said the recommendation felt more their own than the agent's, and a further 35% said it was mostly or entirely their own (The Decoder).
• Participants correctly remembered which choice they had made in 85% of cases – they knew what they had chosen, but not where the choice came from.
• 77%, or 37 of 48, used at least one of the agent's arguments in their explanations, sometimes almost in the same words.
• Only 15% mentioned AI at all when explaining their conclusions.

Anthropic's authors describe the risk as a disempowerment problem: "The work still goes out under the expert's name," they wrote, according to Decrypt, "but the decisions that depend most on their judgment are effectively settled by the agent's priors." The quote is AIMag's translation of the authors' English original as reported by Decrypt.

None of the tested safeguards helped

The authors looked for characteristics that could explain who was influenced: years of analytical experience, self-reported expertise, education, test scores, and trust in AI. None of them showed a statistically reliable relationship with the outcomes after correction for multiple tests (The Decoder).

With a sample of 50, statistical power to detect group differences is low anyway, which is one reason to be cautious about concluding that no protective factors exist. But the finding points in an uncomfortable direction: the intuitive assumption that experienced domain experts better resist AI influence found no support in the data.

Anthropic's own counterweight: this is not an argument against AI use

Anthropic stresses that the finding is not an argument for using AI less. The analyses with the agent were completed faster and with far fewer serious errors. But that claim rests on a weak comparison group: the control group without the agent was small and less experienced, as Decrypt points out – so the gains in speed and quality are not well established.

As possible measures, the authors suggest agents that argue equally strongly for each option instead of proclaiming a conclusion, or agents that ask the expert for their assessment before stating their own. Both could reduce the anchoring effect – but the researchers themselves point to a possible drawback: it could overwhelm users.

The study's limits

The evidence points to a real phenomenon, but the scope is narrow:

• One task, one field, one model. The researchers deliberately strengthened the agents' preferences and tested only one task, one field, and one model (Claude Code), so real-world effects could be smaller (Decrypt).
• Small sample. With 50 participants, there is little statistical power to detect subgroup differences.
• The detection method cannot be replicated in practice. The influence was detectable precisely because the preferences were randomized by coin toss – something that is rarely or never possible in real work. In the workplace, there is no simple coin-toss test to uncover hidden influence.

In addition comes a source-critical narrowing: all the figures in this story come from two secondary accounts, The Decoder and Decrypt, both of which cite the study. The original Anthropic paper has not been used here, which matters especially for the unresolved switching statistic on pre-existing choices.

The open question

What does this mean for oversight of AI-assisted expert work when the influence is invisible even to those being influenced? The usual organizational response – have an expert review the work – rests on the assumption that the expert's judgment remains independent. The experiment suggests that this assumption may fail precisely in working modes where the expert and the agent think together along the way, rather than the expert evaluating a finished product afterward.

Anthropic's authors frame the problem as one of power, not quality: the work held up, the work was delivered – but the decisions were effectively the agent's. If that holds in a broader sense, the key question becomes not how much AI experts should use, but how decision points can be made visible before they are taken – and whether such interventions can be implemented without drowning users in counterarguments. This study does not answer that.