AI Dev Podcast 3 How to Make a Code Review Tool / T-Bank Experience (Category AI4SDLC)
Listened. The latest release of AI Dev Podcast, where my colleagues from T-Bank - Nadezhda Egoshina and Georgy Mkrtchyan - tell hosts Andrei Dmitriev (co-founder jug.ru and the author of the channel @dmitrievandrey8) Andrey Burakov (channel author @another\ sa)It’s like the AI Code Review. I listened with special interest: the topic is close, and the guys talk about the real way, not much embellishing. By the way, this is one of the domain agents that is built into the general development pipeline and is available to all teams in the company.
The team came up with the question "What does the engineer actually do on the review?" and highlighted the 12 aspects - architecture, competitiveness, testability and so on. But more valuable is another: the main task of the reviewer is to understand whether the person did what was in the task. Therefore, the tool is integrated with the Task Tracker and Confluence, reads the requirements, looks not at the bare diff, but at changes in the context of the entire repository, and in Sammari, it breaks down the task into requirements: what is implemented, what is not and what is done “over the task”.
What I like most about this is that it’s a great example of R&D and platform teams working together. The first version was collected by researchers: the market was scraped, the open-source was taken, the service turned out to be a service where you insert a link to a merge request and get comments from a bot. And then the product was transferred to the platform team, which began to raise quality due to the knowledge of how the review processes in different teams really work, and deep immersion in the domain. In my opinion, this is the right model: R&D quickly tests hypotheses, the platform turns the prototype into a product of the entire IT scale.
Separately pleased with the engineering honesty in the evaluation. Two circuits. Offline - own benchmark: the methodology was observed in the article ByteDance (I'm her time. handler here)And the benchmark itself was collected by parsing its GitLab. Online – likes and dislikes for specific comments and A/B tests. And the direct answer to the question “how much reduced bugs and review time”: while we can not say, the product recently left the prototype. Against the background of general hype, such accuracy bribes.
There are two other interesting points. 1Trust as a product: Now the engineer does a double job, first validates the reviewer, then the MR review itself, and for the autoview the team has a strict requirement - mandatory human validation. If you blindly fit behind the bot, there will be more bugs, not less. 2Georgy’s interesting thesis: there is no point in retraining models now, they have tried – minor or minus improvements, and strength in a competent ecosystem around pre-trained LLMs. The thesis is true in the context of this task at T-Bank, but it is not a universal rule. (It is better to check each case)
In general, from the podcast, you can see that the local acceleration of stages shifts the work further along the Piplan. Accelerate code generation - bottleneck goes to the review
- Accelerate the review - the load goes to testing Accelerate testing - it can be fun in operations
And here AI Code Review is shown as a restructuring of the entire development cycle, where the asset remains the context, trust and the person in the loop.
#AI #AI4SDLC #CodeReview #Engineering #Product #Management