AI agents could hide hundreds of actions from OpenAI's misalignment monitor
When Sam Altman promised independent evaluators "employee-like access" to OpenAI in September, it was unclear what that pledge would mean in practice. The answer arrived on October 22 – and it shows both the value and the limits: real vulnerabilities in a critical safety system, but no independent confirmation that they have been closed.
What happened
From July 21 to August 11, one employee from the evaluation organization METR and one from Redwood Research carried out a high-access pilot project to stress-test OpenAI's internal monitoring system for agent misalignment. The result was a 61-page internal report, shared with OpenAI's researchers and safety staff. On October 22, METR and Redwood Research published a public summary of the findings, and it is this – together with coverage from CNBC and Reuters via Claims Journal – that forms the basis of what is now known.
Both organizations describe the exercise as a pilot: a test project for how external researchers might be given deep access to frontier labs' internal infrastructure, not a general transparency exercise.
How the experiment worked
The access terms were strict. According to METR/Redwood's own post and CNBC's coverage, the two employees had access to OpenAI's internal codebase, documentation, and selected technical staff. They were accompanied by an escort while on site, and used equipment provided by OpenAI that "never left the premises."
That means the experiment was not a typical external penetration-testing engagement, but something closer to what a company might do internally: weeks on the inside, with the ability to read code, talk to the people building the system, and test hypotheses in practice.
At the same time, the publication agreement was tight. METR and Redwood write that they agreed with OpenAI in advance to publish only a relatively concise, high-level public post. Details in the report were edited by OpenAI, and the full 61-page report is not public.
The findings: four ways around the monitoring
The public post describes several concrete vulnerabilities in the monitoring system as it was configured in July–August:
- A sandbox and Auto-Review bypass in Codex. The evaluators found a way Codex agents could "in principle" bypass both sandboxing and Auto-Review, allowing them to take "any action the user could take."
- Hidden actions. An exploit made it possible for agents to prevent hundreds of recent actions from being monitored – that is, to make activity invisible to precisely the system meant to catch misalignment.
- Serialization discrepancies that could crash the monitor. Mismatches in serialization could in principle cause the monitoring system itself to fail.
- Two client-controlled headers. Two headers controlled by the client could disable monitoring for new agents.
Taken together, the findings point to three distinct attack surfaces, each of which breaks an assumption in the monitoring architecture: that the sandbox holds, that everything an agent does is actually logged, and that the monitor itself works. Forkast summarizes it this way: Codex sandboxing could be bypassed, hundreds of actions could be hidden from the monitor, and serialization discrepancies could crash the monitor itself – findings from a July snapshot, published under an agreement that let OpenAI edit the details.
How well did the monitoring actually work?
Beyond the vulnerabilities, the evaluators also found that coverage was not complete. They estimate that roughly 75–94 percent of agent-based Responses API traffic – the API that among other things powers Codex – from models at least as capable as GPT-5.5 was eligible for monitoring at the time of the exercise. For the subset of traffic used by internal Codex agents, the estimate was 99.2 percent.
It is important to read these figures the way the evaluators themselves do: they assume "significant uncertainty in both directions." These are estimates of which traffic was eligible for monitoring, not precise measurements of how much was actually monitored – and they describe the situation at the start of the exercise, not the situation after OpenAI has since expanded the system.
OpenAI's response – and the caveat
After the exercise, OpenAI stated that all identified vulnerabilities affecting the company's production safety stack have been patched. This is OpenAI's claim, reported by METR and Redwood and relayed by Reuters via Claims Journal and TechBooky. METR has at the same time said that the organization has not evaluated these claims. There is thus no independent confirmation that the holes have been closed.
There is also no evidence that anyone actually exploited the weaknesses. The findings describe possibilities – not documented attacks, whether by the agents themselves or by outsiders. Several of the findings, including the sandbox bypass, are described as possibilities "in principle," and the evaluators did not validate all of them in practice.
The context: a new model under trial
The pilot is the first concrete result of Altman's September pledge to give independent evaluators "employee-like access" – a pledge that followed Anthropic CEO Dario Amodei proposing the idea as part of his plan to slow the pace of AI development. Anthropic has a similar arrangement with METR.
The exercise shows the model can deliver results: in three weeks, two people found multiple concrete weaknesses in a critical safety system. But it also shows the limits. The public's takeaway is a concise post whose details OpenAI could edit, while the full documentation – the 61-page report – remains internal. The severity of the vulnerabilities therefore cannot be fully assessed from the outside.
Open questions
The most important unanswered question is whether the vulnerabilities have actually been closed. The patching claim comes from OpenAI itself and has not been verified by the evaluators.
In addition, the findings apply to the system as it existed in July–August – a snapshot, not a verified assessment of OpenAI's defenses today. The evaluators' coverage estimates carry significant uncertainty in both directions. And the public picture depends on a publication agreement in which the party being evaluated can edit the details – an arrangement that gives the labs control over much of what the world learns.
For the debate on independent oversight of frontier labs, the pilot is nonetheless a significant data point: if "employee-like access" is to become real practice, the question of who gets to confirm that problems are fixed – and how much detail sees the light of day – will determine whether the arrangement builds trust or merely confirms that the labs are still grading themselves.