[1/3] Resolving Code Review Comments with Machine Learning (AI column)

This one whitepaper Google Research, first published in 2023 Presented at the International Software Engineering Conference (ICSE) April 2024 years. The research is devoted to AI-fixing of the code review process. The bottom line is that the researchers have built a system that is able to understand reviewers' comments in natural language and suggest appropriate changes to the code. The authors describe how Google successfully implemented this technology into its development workflow, resulting in significant time savings and improved productivity. I liked this paper with its approach - the authors did not just stick AI somewhere in the development process, but did it in a product style, changing both the model itself and the UX process itself. The essence of the article itself can be described as follows:

- IssuesCode revision is an important process especially at Google, but it takes a long time. Developers spend an average of about an hour on each code change after a review. **- Decision.**The AI system reads reviewers’ comments and automatically suggests code changes to fix problems. **- How it works.**The system is trained on past code reviews; it analyzes the context of the code, understands the reviewer's comment and generates appropriate changes. This solution has gone through several iterations from asynchronous correction generation to the generation of the proposed fix on the fly while the reviewer writes its comment. The final version assumes that the reviewer can agree with the proposed model correction, and can edit it. **- Effect.**The implemented solution eventually led to the fact that the authors of the changes accept the automatically proposed model in 7.5% of cases. This saves hundreds of thousands of developer hours each year. **- Flexibility.**The system handles both simple and complex changes. Separately, the authors note that reviewers began adjusting their comments to the code so that the system generated better clues for the author of the code. - Feedback.: User feedback has helped improve the system over time by filtering out incorrect predictions. - Benefits for developersThe system allows you to focus on the more creative aspects of development instead of routine.

The author's approach is approximately as follows:

• The authors perceived this problem as a standard text-to-text ML problem for the use of which they took a traditional transformer based on T5X Framework The authors took a real code review and made inline comments from the review into the code in the form of comments, and the target was to be able to predict the proposed diff patch, which was actually proposed to solve this comment from the review.
• He trained a model for code review and other tasks in the field of software engineering and used for this purpose DIDACT framework. The training camp consisted of 3 billions of examples of which 60 There are millions of examples of code review. Pre-training used a fairly free set of code review examples, including automatic comments, comments for the entire set of changes, and entire files. For fine tuning, code-review examples were used where only people wrote comments. The standard cross-entropy loss metric typical of these models At the inference stage, the model was used to give the desired precision - each prediction was accompanied by probability and a greater probability meant greater confidence in the prediction model. To validate the model, a data set was used that was not used in training. Separately, the authors set threshhold for predictions, that is, the proposed changes are shown to users only when the model is confident in its prediction and satisfies additional heuristics.

Continuation in next postI will tell you how this solution was built into the review process.

#Software #AI #Engineering #Process #DevEx