[1/2] This Year in Uber’s AI-Driven Developer Productivity Revolution (Filed under DevEx)
[1/2] This Year in Uber’s AI-Driven Developer Productivity Revolution (Rubric #DevEx)
Interesting. speech From the guys at Uber on the topic of developer productivity, in which they talked about their approach to using AI to improve it. More details can be read hereBelow I will tell you the main thoughts of the speech.
Uber is a large multinational company with distributed teams. 156+ million MAUs, 30M trips a day, 10k cities
- The company has an IDP. (internal developer platform)An internal development platform that assists engineers in their work The guys in the platform focus not on tuning, but on the convenience of engineers and focus on NPS. (net promoter score) Uber faces the challenges of tech debt and the need for codebase migration, and they also have a trend of flat headcount and do more with less. As a result, they decided to focus on using AI to increase the productivity of engineers. To solve these problems, a separate platform team has been created to work on the AI developer experience, which includes engineers from all platforms + specialists in ML who provide models and abstractions for creating AI-based tools. Uber has been practicing AI hackathons since 2022 And now he is building his own agency. LangChain The focus of AI in SDLC was on three things: coding assistance, generating tests, java to kotlin migrations.
——— Code assistance ———
As a coding assistant, Uber used Copilot, but then there was a hypothesis that you need your assistant with a model trained on your code. For this model set criteria for success: +10% acceptance rate, latency less than 1s 100 Tokens, economic efficiency, as well as the integration of the assistant in work processes. The project included the development of MVP and evaluation of the effectiveness of various AI solutions. The project required a lot of resources and time, but within six months the MVP was ready. Further, the authors evaluated the results and decided to stop the project with such conclusions.
- MVPs are easy, productionisation is hard
- Latency requirements vary per tool
- User Experience matters
- UI surface cannibalization is a risk
- Follow ecosystem principle
- Continuously evaluate landscape
- The conclusion was to use industrial tools, but were able to reuse from their MVP parts of the project. -- Code context gathering - Summarize & rank code context to provide best input to use-cases (data race fixer, linter warning fixer, crash fixer) -- Gather telemetry -- Fine-tuned LLM - Custom model with knowledge of internal libraries, custom frameworks, and company-specific best practices In addition to the code assistant, the guys made an assistant for questions on the database of monorep, which are in Uber. 6:) Also, the guys conduct workshops to teach engineers the best practices of using AI, as well as actively evangelize the approach inside. In the future, they plan to add Integration with platforms such as VS Code and Xcode. Vendor fine tuning and additional features to expand the functionality of the assistant Use of tools for analytics and workflows
Continuation of the review in next post.
#Management #Leadership #Software #SoftwareDevelopment #Metrics #Devops #Processes #AI #ML #DevEx