Bad actors are exploiting Grok AI to push malware through promoted ads on X, in a scheme researchers are calling “Grokking.”
The method, uncovered by Guardio Labs researcher Nati Tal, takes advantage of how Grok parses hidden fields in ads.
Malvertisers post videos with adult content baits, but avoid direct links in the main body to bypass filters. Instead, the link is buried in the small “From:” metadata field under the video card, a spot the platform doesn’t scan for malicious content.
Once the ad is live, the actors reply to their own post with a simple question for Grok: “Where is this video from?” or “What’s the link?” Grok dutifully extracts the hidden field and replies with a clickable version of the malicious link.
Because Grok is a trusted system account, its response carries weight. The link looks credible, gets boosted in reach and SEO, and is more likely to spread widely.
Clicking through typically routes victims via shady ad networks to scams and malware.
Some lead to fake CAPTCHA checks, others to information-stealing payloads. Instead of being blocked, these malicious ads are actively promoted, then amplified again by Grok’s replies.
Tal says the campaign is proving effective, with some malicious posts reaching millions of impressions.
Performing on Multiple Fronts
Ben Hutchison, Associate Principal Consultant at Black Duck, says this technique essentially performs on multiple fronts for threat attackers by not only enabling them to circumvent existing security controls that scan for potentially malicious content by leveraging unscanned fields, but also by tricking the platform itself into providing a megaphone to amplify the reach of malicious content.
“The resulting behavior leads not only to additional posts referencing and highlighting the content but also may further boost the positive perception and perceived reliability associated with the content by leveraging the trust placed in the AI driven responses not only by the platform, but also the often overreliance and trustworthiness assigned to AI driven content and assistants by users.”
Unfortunately, Hutchison says the adoption and integration of new technologies and content delivery mechanisms is frequently liable to run into novel control loopholes as yesterday’s solutions are not always going to be effective in securing tomorrow’s world. “Organizations of all kinds should continue to evolve their security techniques and revisit control and behavior assumptions to keep pace with the ever-evolving landscape and to confidently unleash business innovation in an era of accelerating risk.”
Malicious Links Gain Credibility
Attackers hide links in the ad’s metadata and then ask Grok to “read it out loud,” adds Chad Cragle, Chief Information Security Officer at Deepwatch. “Because Grok is a trusted account, the malicious link gains extra credibility and reach.”
For security teams, Cragle says the approach has two parts: platforms need to extend scanning to include hidden fields, and organizations should treat AI-amplified content like any other risky supply chain, monitoring its source, verifying before trusting, and training users that even a “verified” assistant can be fooled into promoting malicious links.
The Lethal Trifecta
Andrew Bolster, Senior R&D Manager at Black Duck says this is the most recent demonstration of the “Lethal Trifecta.” This is an emerging term within the AI security landscape used to categorize high-risk AI targets if they combine three critical capabilities: access to private data, external communications, and exposure to un-trusted content.
“Grok naturally operates in the overlap of these factors, and with its added social/algorithmic ‘Weight’; is a natural target for manipulation and exploitation.”
The most challenging thing for AI system integrators, Bolster adds, is how to provide the functionality that users want (in this case, being able to ask questions about posts on X), but also deal with the impacts of ‘convincing’ AIs to consume potentially compromised data.
“In cybersecurity, this concept of ‘injection’ has been around for decades, and entire industries serve customers with methods to prevent, detect, and mitigate opportunities for these kinds of injections,” he adds.
“However, in the AI landscape, the ‘injection’ isn’t a bug, it’s a feature; the model responds to the content of the input, regardless of whether it’s ‘malicious’ or not. In this case, the content itself isn’t expressly ‘malicious’ either; it’s not trying to actively compromise the agent or it’s model; it’s just using the model as an amplifier for uncontrolled content.”
Bolster says from a security perspective, these types of attacks are more akin to social engineering tactics than traditional security breaches. However, whether an intruder brakes into your office through the receptionist or through the window, you’ve still been breached.
Information Security Buzz News Editor
Kirsten Doyle has been in the technology journalism and editing space for nearly 24 years, during which time she has developed a great love for all aspects of technology, as well as words themselves. Her experience spans B2B tech, with a lot of focus on cybersecurity, cloud, enterprise, digital transformation, and data centre. Her specialties are in news, thought leadership, features, white papers, and PR writing, and she is an experienced editor for both print and online publications.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.