Slide contents

1. 1. AI made development faster

   How can we tell if that is true?

2. 2. What did the measurement find?

3. 3. Perceived speedup needs a separate check

4. 4. METR’s result comes from familiar repositories

5. 5. CLI agents: more merged PRs

   Counts PRs, not time or quality

6. 6. Commits per engineer rose at Anthropic

7. 7. Each result applies to its own conditions

8. 8. Alexander Polomodov

   Alexander Polomodov, Technology leader with experience as a CTO and Technical Fellow at a large fintech, polomodov.tech

   Research Insights Made Simple

   Connecting research with engineering practice.

   Focus — Productivity · architecture · AI4SDLC

9. 9. Two levels of measurement

10. 10. 1A. Company-wide measurement

    Discovery + delivery · the path to a customer outcome

11. 11. From hypothesis to customer value

12. 12. How to measure the end-to-end outcome

13. 13. Product engineers own work end to end

14. 14. A tiny team owns the whole cycle

15. 15. 1B. Company-wide delivery

    Measuring AI code assistants and agents · DX and RIMS #11

16. 16. How DX measures AI in delivery

17. 17. Measure adoption in the context of work

18. 18. An adoption rate needs a denominator

19. 19. Assistance mode changes the work

20. 20. A tool event is not an outcome

21. 21. Google · Measuring Developer Goals

22. 22. Session durations are not human effort

23. 23. Local speed changes downstream queues

24. 24. Logs and survey form one picture

25. 25. Three methods reveal different aspects

26. 26. New tools change measurement itself

27. 27. The comparison sets the strength of inference

28. 28. 02. The economics of specific use cases

    AI review · migrations · interface prototypes

29. 29. Each use case has its own unit of value

30. 30. AI review is already enabled

31. 31. A useful comment has a cost

32. 32. Review settings change the cost

33. 33. Tune AI review by PR risk

34. 34. Repetition drives migration economics

35. 35. A migration ends at acceptance

36. 36. Setup pays off at the fifth batch

37. 37. The remaining migration shapes the decision

38. 38. PMs can test an interface earlier

39. 39. Measure the cost of a product decision

40. 40. One test scenario costs $160 less

41. 41. The prototype carries a decision into delivery

42. 42. Gains show up in time, money and risk

43. 43. What to change after the calculation

44. 44. Continue the conversation in the podcasts

45. 45. The next tasks may add less value

46. 46. Measure the change in work

    Discovery + delivery · use-case economics

    Book Cube

    Which decision can you make after measuring?

    Alexander Polomodov, Technology leader, polomodov.tech

    t.me/book_cube