Trump's AI pact calls itself "almost a constitution" – but is voluntary and without oversight

Trump's AI pact calls itself "almost a constitution" – but is voluntary and without oversight

This past week carried both halves of the same story: At Tech Week in San Francisco, founders and investors lined up in full agreement that the White House's hands-off line toward the AI industry is good news. The same week, three OpenAI safety researchers said they were fired for prioritizing safety — something the company denies. The contrast makes the question concrete: Can the labs police themselves, as the voluntary pact of September 29 assumes? (AP, Al Jazeera)

What happened

President Donald Trump gathered tech titans in Washington late in September for a lunch and a ceremonial signing of a voluntary safety pact, which he described as "almost a constitution" around "Super Intelligence" — the administration's new preferred term for AI. The pact was signed on September 29, and during the week of October 6–10, AP reported from Tech Week in San Francisco on the founders' enthusiasm for the line — in short: don't regulate. (AP)

On Thursday October 8 or Friday October 9 came the counterpoint: Safety researchers Mikita Balesni, Tomek Korbak and Jasmine Wang went public after being dismissed the week before, claiming they lost their jobs for prioritizing safety. They warned of a "chilling" effect on other employees who are concerned about the risks of frontier technology. OpenAI says the decisions had nothing to do with safety concerns. Neither party's version has been independently verified. (Al Jazeera)

What the pact actually contains — and what it doesn't

The agreement is 308 words long. It states that advanced AI labs should follow safety protocols and establish internal teams that monitor their own progress. That is all. The administration has so far not given details on how it will prevent what OpenAI CEO Sam Altman has called "bad things" from happening, according to AP.

Three caveats are important. First, the pact is voluntary — no enforcement mechanisms are given, and it is not law. Second, we know its contents only through AP's account; the document text itself is not available in the accessible source material. Third, the list of signatory companies is known via Al Jazeera, which names OpenAI, Anthropic, Google, Meta, SpaceXAI and Nvidia — but Al Jazeera's text is truncated, so the full terms of the commitments cannot be verified.

Why the market is cheering

At a packed event during Tech Week on Wednesday, safety challenges were far from the top of the agenda. Founders and investors said the government's hands-off line would yield higher profits for companies building AI-powered defense and security systems, AP reports.

Kyle Stanford, VC research director at the analytics platform PitchBook, put a number on the enthusiasm: "I want to start with one number that I think really underscores the investment opportunity in defense, and that's $1 trillion. The US government is for the first time allocating $1 trillion to its defense budget. And $13.5 billion of that will go to autonomous systems and AI."

Note that this is an analyst's characterization as reported by AP — not figures verified against budget documents. But it explains the mood: With defense budgets of that magnitude and no federal oversight line, investors see a growth market with little friction.

The policy behind the pact

The pact does not stand alone. Trump rescinded a Biden-era AI executive order on the first day of his second term — an order that was, among other things, meant to guard against the risks AI systems pose to consumers, workers and national security. The administration has since steered federal policy away from oversight and toward rapid growth, including by canceling earlier requirements that AI developers share safety test results with federal authorities. In December 2025, Trump signed an executive order aimed at blocking AI regulation at the state level, on the grounds that a patchwork of rules could suffocate the industry. (AP, AP/MSN)

The name change is political too. After the meeting with industry leaders at the end of September, Trump signed an executive order requiring federal departments and agencies "to the maximum extent permitted by law" to use "'Super Intelligence' and 'SI' instead of 'Artificial Intelligence' and 'AI'" in official correspondence and public communication. The order does not apply to private companies, and Altman has said OpenAI has no plans to change its name: "No plans to do that. But we have talked a lot about superintelligence, and we will continue to talk about it," he told Politico's Decoded podcast. (Business Insider)

The context for the pact also includes a public squabble the previous month: An Anthropic engineer resigned and demanded a pause in technology development to prevent "superhuman" systems from wresting control, and the heads of Anthropic and OpenAI echoed calls to slow the pace. Trump responded on Truth Social that "the only control or 'guardrails' AI needs is a STRONG AND SMART (high-IQ!) PRESIDENT." Critics have, according to AP, interpreted the leaders' pause manifestos as meant to curry favor with investors ahead of stock listings — these are attributed views, not established facts.

The counter-evidence the same week

Balesni, Korbak and Wang made their claims in an open letter on Thursday. "Given the significant safety concerns around the development of AI, employees must not be left in a work environment where fear and unclear rules inhibit AI safety work and undermine third-party accountability," they wrote. (Al Jazeera)

OpenAI said in a statement: "We want to be very clear that these decisions were not about raising safety concerns or speaking out." The company cites a significant breach of trust. The researchers claim they were fired for prioritizing safety; the company says that is not true. The sources contradict each other, and neither side has been verified. The conflict is unresolved.

The episode nevertheless lands squarely on the pact's core assumption: that labs can monitor themselves. One of the open questions is precisely whether internal monitoring can work if employees who say they raised safety concerns are simultaneously disputing why they lost their jobs.

The criticism of voluntarism

David Robinson, former head of OpenAI's safety team, gave voice to the skeptics' side: "This is no substitute for hard law, and it gives the companies a lot of freedom," he told AP. "The time between model generations is measured in weeks … and I think we have some very serious safety challenges ahead of us in the nearer term."

From another direction comes an opposite concern. SoFi CEO Anthony Noto told Politico's Decoded newsletter: "There's no real framework in place" — that is, businesses that use the technology lack a predictable regulatory basis. The criticism of the voluntary approach thus comes both from those who think it is too soft, and from those who think the uncertainty is costly. (Politico)

The open questions

Three questions remain after this week.

The first: What does the pact actually commit anyone to? At 308 words, with no enforcement details and no publicly available document text in the source material, we know that it calls for safety protocols and internal monitoring teams — but not what that means in practice.

The second: Who is to measure whether labs are following it? No oversight arrangement has been given, and the administration has at the same time canceled earlier requirements to share safety test results with federal authorities.

The third: If labs cannot police themselves, who can? The OpenAI dismissals this week are the clearest recent example that this question remains unanswered — and both parties to the dispute have yet to have their version confirmed.