What Happens When Technical Debt Vanishes? (Category #Management)

Imagine a magic wand that permanently removes an entire class of technical debt. Migrations run themselves, and obsolete feature flags disappear as soon as they are no longer needed. Engineers no longer have to spend time on this work. What happens to a team's productivity? And to the metrics we use to measure it?

That is how «What Happens When Technical Debt Vanishes?» by Ciera Jaspan and Collin Green of Google begins. I read this paper in the Developer Productivity for Humans series; I had already brought together other research from the series in two posts, 1 and 2. Here the authors immediately propose a thought experiment: the way we eliminate the debt could be anything, whether AI or static analysis. Assume the problem has been solved, then examine the consequences. The charts in the paper illustrate hypotheses, not measurement results.

The setup reminded me of Brooks's «No Silver Bullet». It makes a similar move: suppose we reduce to zero the effort spent on the accidental complexity of software development, introduced by our tools and implementation choices. If that effort accounted for less than 90% of the total, even eliminating all of it would not produce a tenfold speedup. Understanding the problem and designing a complex system still take work. Jaspan and Green ask a different question: suppose the improvement really happened—would our indicators detect it?

The argument then unfolds in several steps

1️⃣ Freed-up time changes the mix of work The authors assume the company will put that time into new features or improvements to existing products. Other forms of debt will arise there. Over time, the organization may return to its usual level of acceptable risk while tackling more work. So the general complaint that “technical debt is holding us back” may return, even though the specific problem really was eliminated.

2️⃣ People's expectations change too After the code is cleaned up, satisfaction with its quality may rise and then fall: engineers get used to it and begin judging the system against a new standard. Here the reason the indicator returns to its old level lies in the point of comparison. Questions about the specific kind of debt that was eliminated, however, should still show the improvement.

3️⃣ Some indicators may not change at all The number of PRs or lines of code need not rise if engineers have switched from paying down debt to other work. Those counters do a poor job of reflecting a change in what the work consists of. This nicely extends their article «All Models Are Wrong But Some Are Useful» (which I have already discussed): a useful effect may fall outside the chosen model.

The authors even consider Revenue per Engineer as a way to connect engineering efficiency with business outcomes. But they themselves note the caveat: it is affected by the market, product strategy, and other parts of the company. It is not suitable for evaluating an individual team or person.

Looking at the whitepaper as a whole, I agree with its conclusions about the limitations of metrics. But I think an important assumption has slipped into its reasoning about companies, one that is not always true: the freed-up time can be invested in useful work. If the next task is blocked by approvals, sales, or the absence of a clear product decision, that chain may break. Lower engineering costs expand the possibilities, but those possibilities still have to be realized.

So I would look at three things: 1) Did working with this specific kind of debt become cheaper? 2) Where did the freed-up time go? 3) What outcome did that make possible?

A metric returning to its previous level is entirely compatible with success. But by itself it does not prove success—otherwise any lack of effect could be explained away by saying we have simply gotten used to the good stuff :)

#Management #Engineering #Software #Productivity #Research #Metrics