The increasing adoption of large language models (LLMs) like ChatGPT and Google Bard has been accompanied by rising cybersecurity threats, particularly prompt injection and data poisoning attacks. The U.K.’s National Cyber Security Centre (NCSC) recently released guidance on addressing these challenges.
Understanding Prompt Injection Attacks
Similar to SQL injection threats, prompt injection attacks manipulate AI outputs to produce unintended behaviors. These can range from displaying unethical content or malware to compromising the AI’s inner workings. For instance, a vulnerability in the LangChain library exploited by attackers was highlighted by NVIDIA’s Rich Harang. Another case saw MathGPT, which converts user inputs to Python code, being maliciously used.
This type of security concern isn’t just limited to backend systems. With chatbots becoming increasingly integrated into consumer-facing services, like online banking and shopping, there’s an inherent risk. Microsoft’s Bing Chat, for example, was shown to be vulnerable when a Stanford student, Kevin Liu, created an injection prompt revealing its internal script.
The Threat of Data Poisoning
Data poisoning represents another significant risk. It involves intentionally skewing the data sources feeding into machine learning models. With AI models scraping vast portions of the internet for training, they’re exposed to changing, inaccurate, or even malicious content. Research by teams from Google, NVIDIA, Robust Intelligence, and ETH Zurich unveiled two poisoning techniques: ‘split view data poisoning’, which takes advantage of ever-changing internet content, and ‘front-running attack’, exploiting temporary content edits on platforms like Wikipedia.
NCSC’s Recommendations
To combat these vulnerabilities, the NCSC emphasizes a comprehensive security approach:
1. Rigorous Input Validation: Consistent validation and sanitization to counter malicious inputs.
2. Safeguard Against Unsafe Libraries: Opting for safer serialization formats and avoiding inherently insecure ones like Python’s Pickle library.
3. Trust Verification: Ensuring data and software packages come exclusively from verified and trusted sources.
The central message from the NCSC is that as businesses and services become more reliant on AIs and chatbots, there is a paramount need to prioritize rigorous cybersecurity measures.
Keep an eye on our information security news updates as we continue to monitor “Cybersecurity Concerns In AI” and check the security experts respond to this news.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.