Measuring Developer Productivity With DX Core 4

The episode opens on a white paper from the DX team — a platform that measures engineering productivity for a living — and Alexander names the framework's ancestors: DORA, SPACE, DevEx. Evgeny puts it more bluntly: it is like standards, where a new one is issued to fix the previous ones; it has novelty of its own, but much is borrowed from its predecessors. Alexander adds a pragmatic motive: the authors do have deep experience, but their own framework is what lets them feature-fill their platform under their own brand. Evgeny dates the lineage to Accelerate, the book that popularised the DORA metrics: that is where he first met them, and as he recalls it he reread it half a year ago with different eyes. What he valued in DORA was the reference model: a scale for calibrating yourself against companies deploying almost a hundred times a day, and a goal of reaching the elite band.

Alexander points out where the numbers came from: DevOps surveys, with reports from roughly 2013 to 2017, by his recollection, feeding into the book — and the chapters he wants to reread are those on survey design. He follows with a joke about a sixty-year-old patient told by his doctor to tell the stories his neighbour tells: any survey leaves a gap between the honest answer and the socially acceptable one. A recent example: an industry report on platforms in the Gen AI era where, by his reckoning, some six hundred respondents reported in unison that everything was already adopted. From the same book he pulls the link between deployment frequency and lead time: the authors point to lean practice and small batches, but never devised a question about batch size, so they took the inverse indicator — like a sausage, the more often you cut, the thinner the slices. Evgeny notes DORA is cheap to start measuring: Git, pipelines and a tracker are already in place. Alexander recalls promising a conference prize to anyone who could expand DORA's four letters — almost nobody could.

DORA's problem, for Evgeny, is its audience: these are metrics for technical leaders, while lead time tells a chief executive nothing; and they do not measure people at all. Hence SPACE — a hybrid where satisfaction and wellbeing are counted with NPS-style questionnaires that still have to be designed, while communication and collaboration are caught indirectly: his company uses Jellyfish, which looks at who reviews whose pull requests, who answers whom in chat, and how much of a sprint goes to meetings. Alexander calls such analytics exploratory: his colleagues built a graph of interactions and saw an umbilical cord between backend developers and QA engineers — hence the idea of handing quality ownership to the backend and making the tester cross-functional. Metrics, in his view, should be introduced against a concrete problem, as in observability, not because a framework was mandated; Evgeny agrees two or three are often enough. He has designed two platform teams himself: heavy collaboration with stream teams at first, then less and less, the mature part living as a service; a cheap indicator is the request channel and the meeting count.

DevEx came last, and with it DX gained a product: DORA and SPACE were sold as research with benchmarks, whereas the platform around feedback, cognitive load and flow state plugs into a company's systems for dashboards and a market benchmark — where the company sees its competitive edge. A consultant, Igor Kurochkin, whose company is named after a Team Topologies term, chided Alexander about half a year ago for falling behind: the platform had grown its team and shipped adapters for GitLab, GitHub and the rest. Core 4 itself Alexander first took for a rebranding. The announcement highlights multi-dimensional data, levels from the individual engineer to the organisation, a rollout in weeks, and protection against gaming — otherwise motivation shifts to looking good inside the scoring system, and Goodhart's and Campbell's laws kick in: a metric that becomes a target stops being a good one. Data comes from systems and surveys, topped up with interviews with some engineers and free-text fields that sometimes carry a cry from the heart; the advice is to start with baselines, low-hanging fruit and careful communication.

Core 4 is named after its four dimensions — speed, effectiveness, quality and impact — and impact is the one Evgeny considers the authors' find. Each dimension has a key metric, which, as Evgeny recalls it, the paper calls the output metric, plus secondary balancing ones: game speed by opening more pull requests and a balance such as lead time or deployment frequency should sag, making it visible that productivity has not moved. The key speed metric is diffs per engineer, meaning pull requests opened and merged. Evgeny does not recall the details of perceived delivery speed; Alexander explains it as people complaining that the important work moves too slowly. He also supplies a gaming example: if diffs are counted per epic, a dragging epic gets dropped, a new one created and the tasks relinked — after which the metric looks excellent, while a balance such as dropped epics is missing. Extra balances for your own gaming mechanics, Alexander warns, will have to be written.

Effectiveness is measured by a Developer Experience index — survey-based in essence, much like an NPS — balanced by time to the tenth pull request and regrettable attrition: the first pull request can be a readme edit, ten already show what onboarding is like, and regretted departures grow where bureaucracy is heavy. Alexander qualifies that attrition is also driven by the market and money, but one factor is how pleasant it is for engineers to work with what was built for them. For quality the key metric is DORA's change failure rate, and zero is a bad sign too: the team is probably afraid to ship; the balance is recovery time, feature toggles and rollout strategies. Perceived quality Alexander splits into the client-facing kind — bugs from production — and the internal kind, tech debt that makes code harder to change. The impact dimension is the share of time spent on the new: Evgeny recalls an investment-split dashboard that improved Jira hygiene, Alexander a cascading dashboard over a thousand engineers showing more than half the time going into technical work — migrations and architectural change — which you can explain to the business in good time.