Leadership in AI-Assisted Engineering (Category AI)

An interesting talk by Justin Reock, Deputy CTO of DX, which Atlassian recently bought for $1 billion. Reock is known for measuring and improving developer productivity, making him a prominent voice on AI adoption in development. His talk covered engineering productivity and referred to many studies I have discussed before. Here are the main points.

1️⃣ The AI productivity paradox: enormous variation in results Despite the hype, AI’s actual effect on productivity varies widely. In June 2025, Google CEO Sundar Pichai reported a 10% increase in engineering capacity. That same summer, METR found a 19% slowdown with AI, although developers felt 20% faster. I discussed this in posts 1 and 2, and on the RIMS podcast, where we argued that the study was designed around a particular result and represented a very narrow case.

Across the 300+ companies represented on the DX platform, averages look modest: documentation quality +7.5%, code quality +3.4%, confidence in changes +2.6%, and change failure rate -1%. But some companies see metrics improve by 20%, while others see a 20% deterioration. This seems to depend on how they introduce AI and measure its effects.

2️⃣ Psychological safety is the foundation of AI transformation Reock recalled Google’s Project Aristotle, now almost 15 years old. Google studied teams to ask, “What makes a team effective at Google?” Psychological safety was a key answer; I have discussed the research before. It matters for AI adoption too:

• Fear of replacement by AI can lead people to resist adoption.
• Without trust, people do not experiment with tools.
• Communication needs to be transparent: AI complements people rather than replacing them.

3️⃣ W. Edwards Deming: 95% system, 5% people Deming argued that the system — processes, tools and culture — determines 90–95% of performance, with the remainder depending on individual qualities. Applied to development:

• Performance reviews and individual coaching address the 5%, wasting human talent when treated as the main solution.
• The focus should be on improving the system: infrastructure, workflows and developer experience.
• When something goes wrong, look for problems in the system rather than blaming people.

4️⃣ DX AI Measurement Framework: three dimensions of success The speaker described measuring AI’s effects along three axes:

• Utilization
• Impact
• Cost

I covered the approach in an earlier analysis and in my podcast with Zhenya Sergeev. Reock’s point is that a baseline of standard developer productivity measures, such as DORA and SPACE, matters more than AI-specific metrics: it shows the actual effect on outcomes.

5️⃣ Integrate AI throughout the SDLC, beyond coding For most organisations, writing code was never the bottleneck. Constraints often lie in finding information and context, delayed code reviews, coordination, incident response and legacy code. The speaker offered examples from Morgan Stanley, Zapier, Spotify and Booking.

6️⃣ Top 5 priorities for leaders

1. Reduce fear of AI through transparency.
2. Invest in employee education and allocate time for experiments.
3. Introduce measurement and open discussion of metrics.
4. Remove barriers to use, taking a creative approach to compliance.
5. Create feedback loops for continuous improvement.

#Software #Engineering #Productivity #DevEx #AI #Management #RnD #Leadership #Economy #Metrics