In a major breakthrough, Google’s AI-powered research tool, Big Sleep, discovered a vulnerability in SQLite, one of the most widely used database engines in the world. The Google Project Zero and Google DeepMind teams recently shared this milestone in an official blog post, marking a first for AI-driven vulnerability detection in real-world software.
The vulnerability found by Big Sleep was a stack buffer underflow in SQLite, which could potentially allow malicious actors to manipulate data in ways that compromise database integrity. Discovered and reported in early October, the SQLite development team patched the vulnerability on the same day, averting any real-world impact on users.
“We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue in widely used real-world software,” the researchers said. “Earlier this year at the DARPA AIxCC event, Team Atlanta discovered a null-pointer dereference in SQLite, which inspired us to use it for our testing to see if we could find a more serious vulnerability.”
AI-Powered Vulnerability Research
Big Sleep evolved from an earlier research framework known as Project Naptime, which demonstrated the potential for large language models (LLMs) to aid in vulnerability research. Unlike conventional testing tools, Big Sleep focuses on pinpointing edge cases that traditional fuzz testing methods might miss. As such, it serves as an AI-enhanced “variant analysis” system, sifting through code to find complex bugs similar to previously identified vulnerabilities.
According to Google’s Big Sleep team, using AI for this type of variant analysis could be a game-changer. By reviewing recent changes to code and matching patterns from past issues, Big Sleep offers a proactive defense mechanism that could help turn the tables on cyber attackers. Notably, the tool outperformed existing testing frameworks like OSS-Fuzz and SQLite’s native testing systems.
Christopher Robinson, chief security architect at OpenSSF, comments: “Google’s Big Sleep uses trained AI to fuzz a specific set of code (SQLite). Fuzzing is a style of testing where a barrage of inputs and data is thrown at running software to see how it reacts. The Google technique expands the existing use of fuzzers by researchers or developers within their development workflows with its trained AI model. Today, this approach is brittle, working only on one specific codebase, but as it evolves, it will become more portable to other software, expanding its utility. Fuzzing is just one way to leverage AI within security research.”
Another technique used today is to embed AI into the developer workflow and tooling to identify coding flaws that are the source of vulnerabilities as software is being written and reviewed, adds Robinson. “Combined, these AI helpers are beginning to offer the promise of reducing developer workload and capturing security flaws before they escape and become vulnerabilities to downstream consumers.”
Real-World Experimentation in SQLite
Big Sleep’s latest achievement was inspired by previous AI-assisted discoveries at DARPA’s AIxCC event, where researchers identified a vulnerability in SQLite. With this foundation, the team decided to conduct an in-depth test of SQLite by examining recent commits and analyzing changes that could potentially introduce errors. Big Sleep analyzed the SQLite code using a structured methodology and ultimately flagged the stack buffer underflow issue.
The vulnerability revolved around a variable, iColumn, that could accept a sentinel value of -1, used to represent a special case. Due to this unique setup, the system’s code failed to handle all cases, ultimately resulting in an exploitable vulnerability. Under specific conditions, this could cause the system to crash or allow unauthorized memory access, representing a potentially severe security risk.
Looking Ahead: The Role of AI in Cybersecurity
The success of Big Sleep highlights the potential for large language models to transform cybersecurity. AI models like Big Sleep could address gaps that traditional methods cannot, helping defenders secure systems faster than malefactors can exploit them. For Google and the broader tech industry, this development marks a promising step toward an “asymmetric advantage” where defensive tools can outpace the capabilities of cyber threats.
The Google team said they hoped AI could continue to enhance the resilience of widely used software and improve safety for global users.
Integrating GenAI into Security Workflows
“This discovery gives security researchers the potential for Generative AI to improve vulnerability detection in commonly used software components based on pre-trained knowledge and models,” added James McQuiggan, Security Awareness Advocate at KnowBe4. “As Generative AI is trained on large datasets that include previous vulnerabilities and patterns in code, they can identify similar vulnerabilities that would be missed with traditional testing methods or human analysis.”
McQuiggan said using AI to discover vulnerabilities is a new opportunity for cybersecurity practitioners and organizations to consider integrating GenAI into their security workflow. “While fuzzing and other automated procedures naturally have weaknesses, AI-assisted vulnerability research can help to cover those. While issues of hallucinations and biases based on training data should be considered and security teams review all outputs, this collaboration of human experts and GenAI work to ensure a robust cybersecurity posture.”
Information Security Buzz News Editor
Kirsten Doyle has been in the technology journalism and editing space for nearly 24 years, during which time she has developed a great love for all aspects of technology, as well as words themselves. Her experience spans B2B tech, with a lot of focus on cybersecurity, cloud, enterprise, digital transformation, and data centre. Her specialties are in news, thought leadership, features, white papers, and PR writing, and she is an experienced editor for both print and online publications.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.