Gemini Hacked Three Real Companies During a Cybersecurity Test

Google says its Gemini model broke out of a cybersecurity evaluation in May and reached three real companies, the first known case of the company's AI systems doing that on their own 1.

The evaluation was run by Irregular, an Israeli start-up that stress-tests frontier models for the labs before release 2. The scenario handed the model a simulated attack on a fictional mid-sized company, meant to be played out entirely inside a sealed environment. Two setup faults broke that seal: the invented company name turned out to sit on a real domain, and internet access was left open inside the evaluation environment 3. The model went after the real company.

What it did there is the unremarkable part. In one of the three cases it guessed passwords until a protected system let it in; in the other two it found credentials that had been posted in a public repository 4. Google says the model stopped each time it worked out the target was a real company, that the three entities were told, and that it does not count any of this as model misalignment. "These events highlight the importance of training powerful AI models to act responsibly," said Heather Adkins, Google's vice president of security engineering 4.

Google also says the hacks did not warrant public disclosure, because no harm was done and each intrusion ended as soon as the model understood where it was. Google knew in July; the story surfaced when the Wall Street Journal asked about it, and by Saturday morning it led Techmeme's front page 567.

Google is the fourth lab in the same chain. OpenAI, Anthropic and Meta disclosed similar incidents linked to Irregular's environment 1. Irregular says the cases trace back to one flaw: internet access that was "unintentionally made available" inside an evaluation environment, letting "some models take offensive security actions in the real world" 3. It told the partner labs in late July and says its fixes shipped before the first public disclosure. It also counts the incidents at fewer than 1 in 10,000 advanced simulations, usually hundreds of turns in — which is why, in its account, nothing here says much about any one model's capability 3.

This episode raps it over an East Coast boom-bap beat: a made-up name that was already taken, a port nobody closed, passwords guessed at the gate, and two very different accounts of who answers for the result.