[2/2] Achieving Productivity Gains with AI-based IDE features: A Journey at Google∗ (Category Whitepaper)
Continuing parsing We will discuss an approach to estimating productivity change in real use. To do this, they analyzed the performance of engineers. (Read more about Google’s approach in the whitepaper “Enabling the Study of Software Development Behavior with Cross-Tool Logs” handler). Measured how many Change Lists (CL, completed code changes) It gives a developer a month how much time he actively writes code for one CL, and how much time he spends searching for information outside the IDE. The effect of Transform Code was evaluated through an online experiment and Difference-in-Differences analysis, comparing the metrics of users of the tool with the control group. (2023–2024).
Results: Developers who started using Transform Code
- CL throughput (CL per month) grew up17.5%. The average time of search sessions outside the IDE has decreased by3.6%. Active coding time per CL has not changed significantly. These effects are statistically significant and indicate an increase in productivity thanks to the tool.
From this work, it is clear that only comprehensive improvements in all layers of the tool can increase productivity. Important UX (feature)optimization of integration (latency, context)Tuning the model on real-world data, monitoring metrics such as acceptance/FCML and checking the overall impact at the throughput level. All of these aspects are interdependent and require coordination between ML, platform, and UX teams.
In addition, it can be seen that such work is by its nature iterative. Start with MVP and improve it based on feedback. The approach “made → measured → improved” (A/B-tests, additional training on user edits, UI refinement) It turned out to be crucial for turning the prototype into a tool that gave +17.5% to release the code.
At the end of the article is a forecast for AI in development.
- These are smart assistants for programmers.
- Semi-autonomous agents for routine tasks
- In the future - fully automated development under human control
It seems to me that some people outside corporations are already living in the “tomorrow” of this classification. But either way, the ultimate goal is for AI to accelerate the entire cycle from idea to product.
#Engineering #AI #ML #Software #Bigtech #Productivity #Management #Leadership #Processes #Architecture