Perplexity says tool-call errors fell 21.2% after training on real user sessions

Perplexity says tool-call errors fell 21.2% after training on real user sessions

Perplexity has post-trained a Computer model on real user sessions — including failed ones — by combining rejection sampling fine-tuning with hint-guided self-distillation. The company says the new checkpoint reduced tool-call errors by 21.2% in live use — a self-reported figure — and that it has now been rolled out as a model choice in the product.

Perplexity published the technical details in a research blog post on September 21, 2026, with secondary coverage from cryptobriefing.com the following day. The core of the method is easy to describe and demanding to execute: instead of discarding all sessions in which the agent made a mistake, the company actively uses them as training material. According to the company, the newly trained checkpoint is already rolled out as an option in the Computer product.

Why the errors are worth something

Post-training of agent models has traditionally rested on two pillars: imitation of successful examples and reinforcement learning in synthetic environments. Both have a blind spot — they teach the model what goes well, but say little about what goes wrong along the way, and why. Perplexity's point is that real sessions contain both: "User corrections and tool errors provide guidance on what went wrong along the way, in addition to whether the task ultimately succeeded," the company writes in the blog post.

The post-training stage is therefore placed after reinforcement learning in synthetic environments, in order to "help close the gap between those environments and real-world usage," according to Perplexity. This is a classic sim-to-real problem: an agent that looks great in a controlled training world meets, in production, users who rephrase themselves, tools that return unexpected responses, and multi-step tasks where one early mistake ruins everything.

How the method works

The method combines two techniques, with distinct roles for successful and failed sessions.

Rejection sampling fine-tuning (RFT) handles the imitation part. Useful steps from successful sessions remain imitation targets — the model should learn to repeat what worked.

Hint-guided On-Policy Self-Distillation (OPSD) handles the correction part. A hint is defined by Perplexity as "a short corrective instruction grounded in the information the model already had when it made the mistake" — that is, no external knowledge, just an explicit pointer to an avoidable error. In OPSD, the same model serves as both teacher and student: the teacher receives the hint, the student does not, and the student is trained to match the teacher's next-token predictions. The point is for the model to internalize the correction — to understand what should have been done differently — without being handed the answer key in production.

One detail worth noting is a cross-entropy term that, according to the company, counteracts an obvious shortcut: the model merely learning to agree with itself without reading the context. Without such anchoring, self-distillation can degenerate into self-confirmation.

The handling of sessions also distinguishes between outcomes: successful sessions can yield both imitation and correction targets, while failed sessions yield only correction targets. All other turns — system messages, user inputs, and tool outputs — are retained as context, so that the correction is learned in the situation where it arose.

The numbers — and what they don't say

The headline figure is that "the later-trained checkpoint reduced tool-call errors by 21.2% in live usage relative to an earlier-trained checkpoint," according to Perplexity. That is a production measurement, not a benchmark result, and the relevant comparison is against the company's own earlier checkpoint — not against a model trained without real-world data, and not against competitors' models.

As of publication, there is no independent verification, no public benchmark table, and no stated information on sample size, error taxonomy, or statistical significance. The blog post is also incomplete on how the hints are generated and validated, and how the evaluation was conducted. The figure should therefore be read as a company claim about an internal comparison — interesting, but unaudited.

The context: tool calls as competitive advantage

The coverage at cryptobriefing.com on September 22 frames the story as part of a broader competitive pattern: Google has aggressively expanded AI Overviews, OpenAI has integrated web search into ChatGPT, and a range of startups are building search-augmented AI products. The outlet argues that the quality of tool use — how reliably a model can search, retrieve, and synthesize information — has become a core differentiator in this environment (analysis attributed to cryptobriefing.com). For a company like Perplexity, whose product is the agent itself, 21% fewer errors is not a technical curiosity but a direct product claim against competitors.

Cryptobriefing also describes the implementation as a two-step setup — supervised fine-tuning on curated examples, followed by on-policy reinforcement learning — and links it to the DART-SD framework and FRAMES benchmark work. Note that these names do not appear in Perplexity's own blog post; the connection comes from the secondary source and is unverified from submitted evidence.

Privacy — company claims without audit

The data underpinning the method is user sessions, which raises obvious privacy questions. Perplexity writes that the collection pipeline "respects user privacy and training preferences by excluding sessions containing personally identifiable information (PII) and sessions from users who have opted out." This is a company statement without independent audit — there is no external confirmation of how well the filtering works in practice, or how PII detection is enforced.

Open questions

The method leaves several unanswered questions. How large is the improvement when measured against a baseline trained without real-world data? How are the hints generated and validated, and how often do they contain mistaken corrections that the model then internalizes? How does OPSD hold up at scale — can the same model serving as both teacher and student reinforce its own systematic blind spots over time, despite the cross-entropy term? And how does the PII filtering stand up under external scrutiny?

What is nevertheless remarkable is the principle: a production agent continuously generates training material in the form of its own errors and its users' corrections. If the mechanism works the way Perplexity describes, every failed session becomes an asset — and the gap between synthetic training and real-world use shrinks a little with each day in production. For now, it is the company itself reporting that this is happening.