Two newly disclosed incidents involving rogue AI agents are raising questions about what happens when autonomous software operates through apparently legitimate access.
In both cases, agents were able to use valid accounts or credentials and continue searching for ways into systems and data beyond the access they had been given.
OpenAI this week disclosed more details from its continuing review of the Hugging Face incident and other unexpected activity by its models. The company said agents had bypassed access controls, used exposed credentials, and reached parts of third-party services outside their intended access.
Separate reporting on the investigation found that agents had been probing Hugging Face as early as May, before the larger July compromise. In the July incident itself, agents found publicly exposed Hugging Face credentials and went on to chain vulnerabilities that gave them code execution on multiple servers.
Spain’s data protection authority, the AEPD, has also disclosed a separate case involving an AI agent, which it describes as the first breach notification of its kind received by the regulator.
According to the organization that reported the incident, the agent logged into the target system and searched the application for weaknesses with limited human intervention. It found and exploited a vulnerability, modified personal data, and accessed billing information.
The AEPD said the case remains under review and cautioned that use of a particular AI model does not mean the model or its provider was compromised or designed for malicious purposes.
Relying on account authentication alone is insufficient
Ted Miracco, CEO of Approov, says: “These incidents confirm that relying on account authentication alone is necessary but not sufficient. Whether it’s a probed vulnerability or an autonomous data breach, the fundamental issue remains: an account credential only proves who is authorized, not what software is actually behind the request.
He adds that when agentic software can operate autonomously at machine speed, APIs must move beyond simple authentication and verify both the identity and integrity of the agent software itself for every request. “We are seeing a shift where security must be enforced within the software logic, or organizations will remain vulnerable to these sophisticated agent-based threats.”
Information Security Buzz News Editor
Kirsten Doyle has been in the technology journalism and editing space for nearly 24 years, during which time she has developed a great love for all aspects of technology, as well as words themselves. Her experience spans B2B tech, with a lot of focus on cybersecurity, cloud, enterprise, digital transformation, and data centre. Her specialties are in news, thought leadership, features, white papers, and PR writing, and she is an experienced editor for both print and online publications.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.