How and Why to Measure Engineering Productivity

In a large technology company the effectiveness of engineers contributes heavily to the effectiveness of the business, yet the usual aggregate indicators describe the situation as a whole and offer no way to read the contribution of individual parts of the organisation. The author separates processes into Run, Change and Disrupt — operational work, building and evolving products, and internal startups or shifts of business model — and settles on the delivery portion of Change as the subject. That choice also rules out whole categories of company: these frameworks grew inside product organisations and were never meant for consulting or outsourcing.

The complaint against older measurement is stated separately. It counted output rather than outcome and judged the individual rather than the system, which pushed teams toward local optimisation: one stage speeds up while the end result stays put. Every framework that follows is read as an attempt to correct those two specific distortions.

DORA is organised around four metrics: deployment frequency and lead time for changes carry delivery tempo, while change failure rate and time to restore service carry stability. Behind it stand the studies of Forsgren, Humble and Kim, the 2018 book Accelerate, and the later move of the work under Google; a large volume of survey data and its analysis confirmed a positive correlation between delivery tempo and service stability. SPACE, published in March 2021, adds satisfaction and well-being, performance, activity, communication and collaboration, efficiency and flow, reading each dimension at the level of the individual, the team and the end-to-end process, and mixes survey answers with process metrics.

DevEx reduces the picture to three areas — feedback loops, cognitive load and flow state — and splits each into perceptions, workflows and indicators; its authors note themselves that perceptions can only be obtained from surveys, and the 2024 paper backs the framework with statistics from the getdx.com platform. Google's QUANTS was put together by engineers working alongside social scientists, rests on the GSM method, and comes with a triage step of its own: before any study begins, the requesting team is asked whether the thing is worth measuring at all.

Commercial products line up along the same split: Cortex builds team scorecards on top of DORA, SPACE and DevEx, DevEx 360 runs as a survey platform, Code Climate leans on DORA metrics and goal reporting, and Pluralsight Flow tracks cycle time and pull request activity; Etsy, Dropbox, eBay, Amplitude, Monzo, P&G and DHL appear among DevEx users. T-Meter, run as a product by Pavel Akhmetchanov, sits on the working-systems side: templated wiki spaces and dashboards over Jira, ranging from simple configurations for a small team to elaborate ones for a large organisation, each template describing roles, work types and classes of service.

The team dashboard shows delivery rate, the time tasks spend in each status so bottlenecks surface, backlog dynamics, and development cycle time measured from the start of work to the release buffer. For a leader the same picture folds into a table across teams: lead time at the 85th percentile, current backlog, backlog-to-delivery and work-in-progress-to-delivery ratios, predictability as the 85th percentile over the 50th, and the monthly balance of task types. The author cautions that mapping the status model is not trivial work, and that metrics handed down as KPIs turn the measure into the goal. His own preference is bottom-up, provided the people on the ground can be drawn in.