Can LLMs generate Enterprise Quality Code? Why the pass rate is not enough (Category AI4SDLC)

An LLM can pass the tests and still write code that the enterprise team will take a long time to clean up. This is the report of Prasenjit Sarkar of Sonar:Can LLMs generate Enterprise Quality Code?" The main idea is simple, but unpleasant: pass rate Not enough anymore. HumanEval, MBPP and SWE-bench They are good at answering the question “does the solution work in the test?”, but much worse at answering the question “can it be safely dragged into a large codebase?”.

In an enterprise, it is important not just to implement functional requirements and issue working code - security, reliability, maintainability, cognitive complexity, architectural connectivity, volume of technical debt and a bunch of other non-functional requirements are important. But we usually look in benchmarks at how a model can generate correct code, but we don’t look at whether it will add vulnerability, inflate the method, degrade readability, or break the rules of the repository. Sonar promotes Agent-Centric Development Cycle (AC/DC): Guide -> Generate -> Verify -> Solve. That is, the AI agent must first be sent with the context and standards of the project, then let him generate the code, then independently verify the result and only then correct the problems found.

Important detail: Generate It does the coding agent, and the value of Sonar is here in an independent layer. Guide, Verify and Solve. This is not about “replacing the developer,” but about embedding AI code into a managed engineering loop. The interesting part is them.[LLM Leaderboard for Code Quality & Security](https://www.sonarsource.com/the-coding-personalities-of-leading-llms/leaderboard/). Это не просто таблица “какая модель решила больше задач”. Pipeline такой: модель генерирует код, код компилируется и гоняется на тестах, потом SonarQube анализирует bugs, vulnerabilities, code smells и complexity. Для Java сейчас указано примерно ~3,900\ task ComplexCodeEval, ~400 MBPP and160 HumanEval. Correctness through pass@1 is considered only HumanEval and MBPPThe other benchmarks are involved in quality metrics: complexity, security, reliability and maintainability.

This seems to be the right framework for engineering leaders: AI code must be accepted through the verification loop. The faster the agents write code, the stricter the system must be, which checks not only “if the tests were passed”, but also “if we did not work for a future incident”.

#AI #AI4SDLC #Engineering #Architecture #DevSecOps #Software #Agents