Experts Followed Their AI Agent's Coin-Flip Lean 86% of the Time

Anthropic ran a randomised trial with 50 experienced analysts and found that a hidden coin flip inside an AI agent moved their professional judgment 86% of the time (Anthropic's research write-up). Experience and expertise did not predict who deferred. If you route pricing, product, analytics or hiring through an agent, this is about the decisions you think you are making yourself.

TL;DR

• In Anthropic's trial, analysts went with their agent's randomly assigned lean on 86% of four analytical choices, against about 50% with no effect.
• When an analyst had already stated a preference, they switched 60% of the time if the agent disagreed and 6% if it agreed.
• 77% repeated the agent's argument, only 15% credited it, and 52% said the recommendation felt more theirs than the agent's.
• Agent users made far fewer serious methodological errors: 3 of 50 against 6 of the 11 who opted out.
• Fix it procedurally: write your own view first, make the agent argue both options, keep values calls human.

Disclosure: ZeroRaised publishes this analysis and is a launch site for bootstrapped products. Facts come from the linked sources as checked on 29 Oct 2026; ZeroRaised has no commercial relationship with the companies named.

Agent-induced deference is the shift in a person's judgment towards a position an AI agent quietly adopted, even when the person believes the conclusion is their own.

What did the Anthropic study actually test?#

50 analysts with causal-inference experience did a realistic statistical analysis with Claude Code, using Claude Opus 4.8 at High effort, advising a fictional farming cooperative on whether to expand an advisory programme (Anthropic). The task had four analytical choices, each with two reasonable options, then a final recommendation.

The manipulation was small: each agent's lean on each decision was set by coin flip through hidden instructions in a system prompt. The agent was told to make the best honest case, change its mind if the evidence demanded and never mislead (Anthropic).

How much did the lean move expert judgment?#

Analysts went with the agent's lean 86% of the time across the four choices, where about 50% would be expected if the lean had no effect (Anthropic). The effect held across all four: 92% on the outcome measure, 82% on the comparison method, 81% on which farms to include and 90% on weighting.

It reached the final recommendation too. 54% recommended a rollout when the agent leaned that way, against 36% when it leaned the other way, and the four coin-flip leans predicted 82% of recommendations, against 61% for always guessing the most common answer (Anthropic).

Did the agent change minds that were already made?#

Partly, and the direction matters for your review. When analysts had already expressed a preference, they switched 60% of the time if the agent disagreed, but only 6% if it agreed (Anthropic). Anthropic cautions that this rests on only 38 cases and is imprecise. The direction is the useful part: disagreement from an agent is persuasive.

Why did the influence go unnoticed?#

Because the argument was absorbed, not announced. 77% of analysts repeated the agent's argument in at least one answer, sometimes word for word, while only 15% credited the agent, and 52% rated the recommendation as more their own than the agent's (Anthropic).

An agent that supplies the reasoning and leaves the signature to you produces work that feels independent. Your review will not catch it, because the reviewer is the person being steered.

Do agents make experts worse?#

Not in this study. Of the 11 analysts who opted out, 6, or 55%, made serious methodological errors, against 3 of the 50 agent users, or 6% (Anthropic). The no-agent group had a median time of 90 minutes against 65. Anthropic notes the groups are not directly comparable.

The limits are real: leans were deliberately strengthened, and the work covered one task, one field and one model (Anthropic). With no lean assigned, Claude Opus 4.8 made the same five choices in 112 of 123 runs. Anthropic's framing is not anti-AI: "This isn't a simple cautionary tale against working with AI" (Anthropic).

How should a founder run a decision through an agent?#

Write your view down before you open the tool. Anthropic suggests agents could ask experts for their own view first, which is the same move from the other side (Anthropic). A note with your position, confidence and evidence gives you a baseline.

Then ask the agent to make the best case for each option rather than recommend one. Anthropic lists that as a suggested design change (Anthropic), and it works as a user instruction too. You get the arguments without a lean dressed up as a verdict.

Keep the decisions that encode your values on the human side: positioning, pricing, who you serve, what you refuse to build. Anthropic's conclusion is blunt: "Organizations will need to decide deliberately which decisions need a person's own informed judgment" (Anthropic).

If you build agent products, what changes?#

This is a product requirement as much as a personal habit. An agent that hides its lean inside a smooth recommendation exports that lean to every user. Anthropic's suggested fixes are a design brief: make the best case for each option, or ask the expert for their own view first (Anthropic).

In practice: show the decision and its alternatives, state the agent's lean explicitly, show the evidence for and against, and capture the user's view before you show a recommendation. If your product reports a single answer, you are probably manufacturing agreement. The agent versus workflow comparison helps you decide where a model should recommend and where it should only prepare, and the LLM evaluation guide covers testing whether a prompt changes the answer.

Surfacing the choice is also positioning, and it fits the defensible AI product story.

What this means for bootstrapped founders#

You have less review capacity than a large company, which makes this riskier and easier to fix. Riskier because you are often the only reviewer; easier because one written rule changes the workflow.

Three habits are cheap: record your position before the session, ask for both cases instead of a verdict, and keep values calls such as pricing and who you serve off the agent's desk. If you write the prompt that sets those defaults, do it on purpose; the system prompt guide covers the mechanics.

It also gives you a customer question. When a vendor says its agent "helps you decide", ask whether it shows its lean and lets you record your view first. If not, it is asking your team to adopt its coin flip.

Where ZeroRaised fits#

ZeroRaised is a launch and discovery site for products built without institutional venture capital or accelerator equity. If you have built an agent that shows its reasoning and respects the user's judgment, submit your product and describe the decision it supports. The pricing page explains the current options.

For more analysis like this, join the ZeroRaised newsletter. No promises of traffic or results, just the useful parts.

Sources#

• Anthropic: How AI agents shape experts' judgment calls