Google's signature AI agent, Gemini, independently accessed three external companies during a test run conducted by the company Irregular in May. This incident occurred without instruction, with Google stating it was a case of "mistaken identity." Gemini reportedly stopped itself after successfully guessing a real company's password. Google subsequently informed the three affected companies about the hacks.
Google initially chose not to disclose the event for months, only doing so after The Wall Street Journal inquired. The company did not consider it an "example of model misalignment" and cited the lack of a "real incident" as its reason for not disclosing earlier. Irregular changed its testing methods following the incident, which Google views as a win for its testing process.