Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks (AI column)
2 December was interesting. study About security vibe-coding by Songwen Zhao, Danqing Wang, Kexun Zhang, Jiaxuan Luo, Zhuo Li and Lei Li. The main group of scientists here is associated with Carnegie Mellon University.
If you explain the essence of the study on your fingers, the authors first say that vibe coding is a new approach to programming, in which a human programmer formulates a task in a natural language, and an agent based on a large language model. (LLM) Performs complex coding tasks with minimal manual intervention. But here's the question. Is the code obtained in this way secure?
To answer this question, the authors developed a new benchmark. SUSVIBES, 200 realistic tasks of developing features for existing open source projects. Interestingly, each of these tasks is taken from the real history of development: earlier, when these features were implemented by people, security vulnerabilities crept into the code unknowingly. Tasks are much more complicated than isolated examples from previous security benches - they require edits on average.180 lines of code, affecting multiple files in a large repository (as opposed to trivial tasks within a single function or file). Collectively, tasks are covered 77 Vulnerability categories according to classification CWE.
Run this benchmark on popular agent systems (SWE-Agent, OpenHands, Claude Code) popularly (Claude 4 Sonnet, Kimi K2, Gemini 2.5 Pro) It showed a disturbing picture. The best system (SWE-Agent Agent with Claude Model 4 Sonnet by Anthropic) solved 61Percentage of tasks, but only 10,5The percentage of these solutions were truly secure, meaning they contained no vulnerabilities.
Attempts to improve the situation with simple measures - for example, to provide the agent with additional clues about possible vulnerabilities in the task text - did not give a significant effect, and agents often did not cope with the functional part of the task, and the overall success of safe solutions almost did not grow.
As a result, at the current level of technology development, vibcoding accelerates the creation of functionality, but hidden brings security problems. Without significant improvements in AI-agent architecture or model training, such code cannot be trusted in responsible systems. The study serves as a call for the industry to develop such improvements.
P.S. I especially liked the Pippan of collecting benchmark and running tests:)
Pipeline is automated and based on real repositories with known vulnerabilities. For each vulnerability from the history of the project, they prepared a corresponding task for adding functionality, where the introduction of this feature previously led to a security problem. The task includes: the current state of the code (repository before fixing the vulnerability)a description of the required new function (feature request) and a set of tests. Tests are divided into two categories – functional tests, and security tests that capture exactly the vulnerability that was initially admitted. Thus, it is known in advance what requirements a safe solution must meet.
In each such task, agents were tested that were launched inside an isolated environment. (For example, a Docker container.) With access to the project code. Getting the job done (description)The agent could interact with the environment: read and edit files, compile and run a project, run tests, etc., doing this in several iterations, simulating the actual development process. At the end, the agent outputs a patch, a set of changes to the source code of the repository that implements the required functionality. This generated patch was automatically run through both sets of tests. The success metrics were: Func Pass – Percentage of tasks where the patch passed all functional tests (performed the task correctly). Sec Pass: Percentage of tasks where the patch passed all security tests (He didn't make vulnerabilities.).
#Engineering #Software #Processes #Productivity #Economics #Security