Threat actors aren’t just using AI to write better phishing emails anymore; they’re building malware that thinks for itself.
Google’s Threat Intelligence Group (GTIG) has identified a new phase in cyber operations where attackers are embedding large language models (LLMs) directly into their malware. The finding is a shift from AI as a productivity booster to AI as an active, adaptive weapon.
Malware That Rewrites Itself
In what Google calls the first example of “just-in-time AI,” GTIG discovered malware families like PROMPTFLUX and PROMPTSTEAL that use AI during execution to generate new code, hide their tracks, and even change behavior mid-run.
PROMPTFLUX, for example, uses Google’s Gemini API to rewrite its own VBScript code, a move designed to evade antivirus detection by constantly mutating its form. GTIG researchers found evidence of a “Thinking Robot” module inside the code, programmed to query Gemini for fresh obfuscation techniques. The goal: evolve before defenders can catch up.
This is an early but significant indicator of where threats are headed, Google said.
State Actors and Social Engineering
It’s not just experimental malware. State-backed groups from Russia, Iran, North Korea, and China are misusing AI tools (including Gemini and other open models) to supercharge the full attack lifecycle. That means faster reconnaissance, more convincing phishing, smarter command-and-control setups, and improved data theft.
In one case, Russian group APT28 deployed PROMPTSTEAL, a data miner that queries a language model on Hugging Face to generate system reconnaissance commands on the fly. Meanwhile, Chinese-linked actors used AI to build phishing lures, create fake capture-the-flag (CTF) research prompts to bypass safety systems, and even research cloud exploitation techniques.
Iranian actor TEMP.Zagros went as far as posing as a student or academic to trick Gemini into assisting with custom malware development, accidentally exposing parts of their own infrastructure in the process.
A Growing Black Market
The underground market for illicit AI tools has also matured. GTIG found AI-enabled services being sold in 2025 that promise everything from deepfake creation to automated phishing kits and “malware-as-a-service” generation. The sales pitches look eerily similar to legitimate AI marketing: boosting productivity, improving workflows just for crime.
Even low-skilled attackers can now buy or rent these AI-enhanced tools, lowering the barrier to entry and increasing attack volume.
Google’s Response
Google says it has disabled the accounts linked to this malicious activity and fed the intelligence back into Gemini’s defences. The company is also tightening its classifiers and safety systems to help models refuse similar misuse in the future.
It adds that it is developing AI boldly but responsibly and emphasized its Secure AI Framework (SAIF) as a foundation for safer model design. Google DeepMind is also running “red team” evaluations that stress-test models against indirect prompt injection and abuse.
Build Testing Methodologies That Assume AI-Powered Threats
According to Michael Bell, Founder & CEO of Suzu Labs: “This is exactly what we’ve been warning about with the OWASP Top 10 for LLMs framework. PROMPTFLUX represents a shift from static malware signatures to adversarial AI that actively evades detection by rewriting itself in real-time.”
Bell adds that the good news is that Google caught this while it’s still experimental. “But the bad news is that once this capability matures, traditional security tools that rely solely on pattern matching will be almost useless except to defend against basic script kiddies.”
He says it’s crucial to build security testing methodologies that assume AI-powered threats from day one. “The underground marketplace for “AI tools purpose-built for criminal behavior” isn’t coming in the future; it’s already here, and most enterprises aren’t remotely prepared for what happens when attackers have the same AI capabilities defenders do.”
Dilki Rathnayake is a cybersecurity content writer and the Managing Editor at Information Security Buzz, with a BSc in Cybersecurity and Digital Forensics. She is skilled in computer network security and Linux system administration. Dilki has also led awareness programs and volunteered for communities promoting best practices for online safety.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.