Skip to content
back to the archive page
#AI4SDLC

AI sped up development: how can you tell? (Series #AI4SDLC)

AI Dev Live has posted the recap of my talk. To get all the recordings and full conference recaps for free, hit the participate button on the site and subscribe to the speakers’ Telegram channels. The button takes you to a Telegram bot that tells you whom to follow and unlocks access after checking.

In my November 11 talk I used an example from a METR experiment. Experienced developers working in their own repositories thought AI saved them time. When measured, the tasks took longer with AI. Meanwhile, Microsoft saw more merged PRs per engineer among early agent users. How can both stories be true, and what should you measure on your team?

What I covered

  • How to calculate AI adoption and whom to count in the denominator when only part of the team has access to agents.
  • How to measure the journey from hypothesis to customer outcome and spot a queue moving to review after coding gets faster.
  • How to account for engineers’ time when one person has several agents running in parallel.
  • How to price a useful AI review comment, an accepted batch of migrations or a hypothesis tested with a prototype.

More material

If you’ve measured the effect of AI, where did you first see a speedup, and did users feel it?

#AI4SDLC #Agents #Engineering #Metrics #Conference

Open video on YouTube