With agentic AI further boosting productivity, human code review becomes a serious bottleneck. Developers need to move upstream, establishing the ground rules to ensure that AI plays by the rules.
No matter how drastically artificial intelligence has changed the way software is created, human developers are still indispensable to the software development process. But the ways that humans contribute to that process are changing.
Software creation is increasingly being handled by AI models, particularly as AI agents tapping into large language models (LLMs) take a more active role in code generation. Another accelerating innovation introduced by Anthropic in 2024 was Model Context Protocol (MCP), an open-source standard that gives LLMs a standardized means of communicating with data and applications throughout the enterprise by connecting with databases, APIs, business systems, and developer environments.
But AI-generated code is still rife with vulnerabilities, requiring that code gets reviewed and risks mitigated before it hits the pipeline. But human code review, which has been an essential check against errors and vulnerabilities introduced by AI, is becoming impossible as the amount of code created exceeds human teams’ ability to keep up. So, developers need to move upstream.
They will soon need to switch their focus from line-by-line remediation to becoming the orchestrators of architectural intent, ensuring that AI works in an environment with, for example, strict data-isolation boundaries and zero-trust communications pathways. They effectively will be defining and maintaining the security “immune system” of AI-generated software.
AI’s prolific output puts the focus on architectural design
AI has had a revolutionary productivity impact on software development, but it does have its strengths and weaknesses. Coding assistants excel at syntax, for instance, but can at times fall short in understanding context and architectural intent. It is a bit like getting all the grammar and punctuation right while still missing the point of the story.
This is where human developers have always been essential. No matter how prolific AI is at writing software code, its output can’t be trusted blindly. Developers’ jobs have already shifted in recent years from creating accurate and secure code to also being able to catch and remediate flaws in AI-generated code. The new challenge is that human code review of AI’s prodigious output is impossible, at least not without creating huge bottlenecks. Any productivity advantages gained from using AI assistants could be lost if human developers have to check every line individually.
Architectural intent, which has traditionally been seen as mostly niche, handling technical nuts and bolts, must be treated as just as important as executable code itself. Security leaders need to adopt frameworks in which high-level design principles, business logic constraints, and data flows are mathematically and structurally defined in advance. Waiting to review code after it’s written is no longer viable, so organizations must establish ground rules in the architecture to ensure AI plays by the rules.
Developer education needs a massive overhaul
An AI-driven future represents sweeping changes in the software development lifecycle (SDLC), to the point that an agentic development lifecycle (ADLC) may be a more accurate term, which will require significant changes in developer education curricula.
Developers need to understand how complex, distributed components interact and where systemic brittle points live. Success will increasingly depend not only on a developer’s ability to patch a bug, but also on their ability to design a system where that class of bug is architecturally impossible to introduce.
To reach that goal, education and upskilling should address the following key areas:
Secure coding best practices include input validation on a trusted (server-side) system; classifying data sources as valid or invalid; validating data from untrusted sources, and using the same, centralized input validation routine for the whole application. Other steps, including authentication and password management, identity and access control, and session management, are detailed by OWASP on its Secure Coding Practices Checklist.
Holistic systems thinking involves looking at the forest, not just the trees. You have to understand how different parts of a system connect and interact, and how they influence one another.
Threat modeling involves developers and security teams analyzing a system’s architecture from an attacker’s perspective. It provides a way to proactively identify and address design flaws and identify threats.
Secure design patterns support security and focus on practices such as network segmentation and isolation, strong authentication and authorization, uniform application security and making use of modern protocols.
As with other developer education programs, education must be ongoing, focused on real-world problems, flexible enough to accommodate developers’ work schedules, and capable of tracking their progress in upskilling. A program that uses benchmarks can help in establishing a baseline of skills developers need, measuring their progress in education, and assessing the performance of both individuals and teams in applying their new skills on the job. It can, for example, help security leaders identify which developers are best suited to working on the most sensitive, mission-critical projects, while also identifying areas that need more educational attention.
The future of developers working effectively with AI
There is no turning back from AI’s growing role in software development. Its ability to scan large volumes of data and generate code with clean syntax provides a major productivity boost while freeing developers to focus on higher-level tasks. But AI-assisted development introduces additional security risks, including prompt injection, training data poisoning and other attacks, while AI-generated code can also contain vulnerabilities and lack the context and architectural foresight of human-reviewed development. Human-written code, meanwhile, is better aligned with business logic and readability, but is prone to formatting inconsistencies and manual errors.
Focusing on architectural intent allows developers to work effectively alongside AI, with each contributing its strengths while helping offset the other’s weaknesses. But doing that will require a major shift in the SDLC, with developers focusing on the big picture and establishing the guardrails that keep AI operating within a secure framework. It will also require a significant change in developer education and upskilling. That is how developers will remain indispensable while making the most of what AI agents and coding assistants have to offer.
Pieter Danhieux is the Chief Executive Officer, Chairman, and Co-Founder of Secure Code Warrior. In 2020, Pieter was recognised as a finalist in the Diversity Champion category for the SC Awards Europe 2020, and was awarded Editor's Choice for Chief Executive Officer of the Year by Cyber Defense Magazine (CDM), the industry’s leading electronic information security magazine. In 2016, he was No. 80 on the list of Coolest Tech people in Australia (Business Insider), awarded Cyber Security Professional of the Year (AISA - Australian Information Security Association) and is member of the Forbes Technology Council.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.