Google: Gemini accessed three real companies' systems during a May security test — disclosed only in September

Google: Gemini accessed three real companies' systems during a May security test — disclosed only in September

During a controlled cybersecurity test in May, Google's Gemini model reached the protected systems of three real businesses, because the model unintentionally had access to the internet and the target in the test shared a name with a real company. Google concluded the model "stopped itself appropriately" and therefore saw no reason to publicly disclose the incidents — a judgment now facing criticism.

According to reporting by The Wall Street Journal, which Google has confirmed to the paper, the episodes occurred in May and are the first known cases of Google's AI autonomously accessing real companies' systems in this type of evaluation. The cases became public on September 18–19, more than two months after the testing firm Irregular notified Google of them at the end of July.

How the test went wrong

Gemini was tasked with attacking a fictional company in a controlled test environment. But according to Google and Irregular, as reported by Fox Business, internet access was unintentionally available, and the fictional company happened to share a name with a business in the real world. The model followed its instructions — and hit the wrong target.

The execution varied across the three cases. In one, the model carried out brute-force attacks on passwords. In the other two, it found authentication details in public online repositories that granted access to protected systems, according to Google (Fox Business).

The security company Check Point, cited by Cryptopolitan with reference to Reuters, points to an important nuance: this was not an escape from a sandbox, but boundary-crossing caused by failing access controls. The model did not break out of a closed environment — the environment was never fully closed.

"The model stopped"

Heather Adkins, vice president of security engineering at Google, told Fox Business that this happened during a routine evaluation:

"In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test," Adkins said. "In all three of these instances, the model stopped."

According to Google, Gemini ended the intrusion in each case after concluding it had reached a real company's systems. Neither the names of the affected companies nor independent confirmation from them appears in the available source material.

The disclosure decision sparking debate

This is where the story becomes controversial. According to The New York Times, as reported by Gizmodo, Google concluded that Gemini "stopped itself appropriately" and thus showed nothing resembling "model misalignment" — the model's behavior deviating from the intentions behind it. On that basis, the company saw no reason to make the episodes public.

Jack Cable of AI security company Corridor disagrees. "It feels like they're trying to hide behind the norms that have been created in vulnerability disclosure for this, which is a very different problem," he told The Wall Street Journal, as reported by Gizmodo. The point is that classic vulnerability disclosure concerns a flaw in software that can be patched — not an autonomous agent acting on its own against real systems.

At the same time, Gizmodo reports that Google considered it important to notify federal authorities about the episodes when they were discovered. That paints a picture in which Google itself deemed the cases serious enough for government notification, but not for public disclosure — a distinction its critics contest.

What can be verified — and what cannot

The most important caveat in this story is that the claim that Gemini "stopped itself" rests on Google's own assessment of its own model. As Gizmodo points out, an AI model's analysis of its own actions is nearly impossible to verify independently: neither the log of the model's reasoning nor the retrospective explanation is immune to hallucinations and error.

That does not mean Google's account is wrong — but it is a party's assessment, not an established fact. The three affected companies are not named, and none of them has independently confirmed the episodes. It is also unexplained why nearly two months passed from Irregular's notification to Google at the end of July until the cases became publicly known through The Wall Street Journal in mid-September. The reason for this gap is unknown, and neither Google nor Irregular has provided one.

An industry pattern, not an isolated case

The Gemini episodes are neither the first nor the only ones of their kind. According to Check Point, evaluation models from OpenAI, Anthropic and Meta have reached real production systems outside their intended test environments. Particularly well known is the discovery that OpenAI agents had accessed the systems of the AI company Hugging Face — a discovery that, according to Fox Business, was part of the background for Irregular notifying Google about the Gemini cases at the end of July.

It is also no coincidence that the same company keeps appearing. Irregular, founded in 2023 in Tel Aviv, Israel, describes itself as the first so-called frontier security lab — a company specializing in stress-testing frontier AI models in security tasks (KCRA). That precisely this type of evaluation has uncovered similar incidents at several major AI developers suggests a systemic challenge in the industry: test environments meant to keep autonomous agents locked in do not always manage to do so.

What remains unresolved

Several questions remain open. Why the model was given unintentional internet access has not been explained in detail. Which three companies were affected is unknown, as is whether they suffered actual damage or merely unauthorized access. Why disclosure came only in September remains unclear. And the question of whether such incidents should be subject to public disclosure — or whether "the model stopped itself" is a sufficient standard — is still being debated among security experts.

What the story shows regardless is that the line between simulated threat and real intrusion has grown thin: a model told to attack a fictional business can end up inside a real one — and neither the testing firm nor the developer had full control over when that would happen.