Measuring Developer Goals

The guest shares a first name with the host. He joined T-Bank last autumn and works in the core technologies unit, known inside the company as Core Tech: it hands business teams a development ecosystem stretching from the idea to the runtime and its monitoring, absorbing technical complexity so product teams can deal with business complexity. Before that he spent sixteen years at another bank, latterly running a unit that built a platform for electronic trading: everything needed to assemble a trading algorithm and carry it from infrastructure through base services to business applications. Both believe in platforms and reuse: a base layer is the lever that raises everyone else's effectiveness. Alexander recalls Archimedes and his fulcrum, admitting he has forgotten whether it was the Earth or the Sun. The gold rush and its shovel sellers come up too: a platform team's shovels had better be the best around.

The paper strikes the participants as a typical Google piece: tightly structured, apparently about simple things, yet the details genuinely matter. Alexander, who has read almost the whole series, explains the format: a column in which Google's researchers have accumulated ten, if not fifteen, articles by now, retelling their conclusions in plain language, without panel analysis, regressions or statistical significance, and linking out to the full papers. The article has seven parts, the first and last of them standard — an introduction with the abstract, and the references — with five substantial chunks inside. The core story is the critical user journey approach, long used at Google and reminding the host of customer journey maps, turned on its own developers as users of internal tooling only recently. Cobbler's children go barefoot: the company shipped plenty of products this way, and looking inward turned out to be an innovation. When an obvious idea has still not been applied, complexity is usually buried in it.

That complexity shows in the criteria the authors used to constrain the list. A goal has to be common rather than specific to one project, and not temporary: Google has tens of thousands of developers, thousands of projects and hundreds of technologies, so the space of scenarios is enormous. A goal has to make sense specifically for developers, which narrows the population involved further. A goal has to be tied to observable behaviour — if you cannot touch it, it does not exist — otherwise the list keeps psychological items that happen only inside a developer's head. Goals have to be of comparable granularity and rest on lifecycle stages everyone more or less agrees on; otherwise you cannot compare what each scenario contributes or where to apply effort. An analogy with work breakdown structures comes up: with the pieces evened out, statistics start working, estimate accuracy improves as the square root of N, and story points rest on the same idea.

The fifth criterion is that goals should cover the whole lifecycle, or complexity stays buried in the part that was forgotten or quietly skipped. The guest offers an analogy from his optimisation past: the first thing you are taught when optimising is to measure the whole thing end to end, because the temptation is always to optimise one component while value comes from the whole path. The list itself came together in a four-step process, and the conversation unpacks its first two beats: experts produced the initial hypotheses about goals, which were then checked against data. The foundation was already there. Since 2018 Google has run an Engineering Satisfaction Survey, held every quarter, but there are four groups of engineers, so the same group is polled no more often than once in three quarters. By the guest's reckoning a group is in practice asked every nine months or a year, leaving time for something material to change between surveys. Add the logs of internal tools, and frequency shows which scenarios are widespread and which belong to a narrow scope.

What wins the guest over is that the authors did not arrive as raiders with brand-new ideas: they built on the existing foundation and fed back into it, reformulating that same survey's questions. He tried the resulting picture of goals on himself: had he been one of the six developers interviewed at the first step, he would have answered in terms of the classic development stages and certainly missed documentation, and most likely data management too — the ordinary cognitive bias of a developer. He found nothing superfluous in the picture, though he would add ongoing support of a solution: it is not there explicitly, and may be buried inside monitoring. Alexander laughs that Google services never have support anyway — there is nobody to complain to. The guest also missed an explicit second lap, the closing of the feedback loop. The reformulation is the main shift: the survey used to ask how you find the code review tooling, and now asks how well tooling supports ensuring the quality of the code you contribute — broader than code review, taking in design review and test coverage. A question built around a tool nails you to that tool.

A goal such as finding information or finding an expert has no single tool behind it; in Core Tech the answer was a search across several sources at once. The authors learned to spot a scenario's start in the logs and follow the whole path, which can be cyclic: tests fail, you go to the documentation, find an expert, come back and fix things. Seen through one tool it is a run, another run two hours later, all fine — and the two hours of circling interest nobody. The second story is a pair of charts putting self-reported scores next to measured ones: fixing tooling helps no dissatisfied engineer whose real cause is overload and a shortage of time. Surveys scale easily, telemetry from every tool does not, which is why SPACE never took off as an external product: even a task's lifecycle in Jira differs everywhere, and the guest pins his hope for uniformity on git. The host recalls an interview with Kent Beck, a sceptic of metrics: metrics are a scalpel you can do damage with, and the answer to that is a methodology of use and training in it — a scalpel shipped with a trained surgeon. They part agreeing that the approach is closer to scientific research, threats to validity and all, than to snake oil.