[2/2] Measuring Developer Experience With a Longitudinal Survey (Filed under DevEx)

[2/2] Measuring Developer Experience With a Longitudinal Survey (Rubric #DevEx)

Continue. firstI will tell you about the main points in how the surveys about developer productivity in Google are arranged.

4. The main features of the survey program in Google are the following points: Each quarter, a new couple is responsible for the survey program: researcher and engineer. The researcher is responsible for the development and evolution of the research and communication program, and the engineer is responsible for the automation of the data processing and analysis infrastructure. The interchangeability of this pair allows you to better document the process and make it impartial and repeatable. The process of conducting surveys is clearly described and built in a pipeline format: identifying changes in surveys, implementing changes, verifying, launching, analyzing, preparing and sending reports. The process is highly automated and most of it goes without human intervention.
5. This process has helped researchers gain insights, such as What was the impact of the COVID pandemic?19 Google developers. Details on this in my analysis "Hybrid Productivity"

• How to validate other metrics, for example, so the guys learned flow, focus and friction in the development - more about it in detail my analysis "Measuring Flow, Focus, and Friction for Developers" How to dramatically improve a metric, for example, tech debt. Details on this in my analysis Defining, Measuring and Managing Technical Debt Research Insights Made Simple #2 whitepaper

6. It is necessary to deal with the evolution of surveys over time, since they tend to lengthen and the response rate may decrease. The authors struggle with this through 2 stuff

• Sampling. The authors beat all engineers into three cohorts, and each cohort is surveyed once in a while. 3 quarter (Only a third of engineers answer questions every quarter.). It is important that the groups 3This allows you to answer questions at different times of the year.
• Reporting. The authors anonymize the results and make them available to all, as well as show what decisions are made on their basis and what results they lead to. This encourages engineers to fill them in.

7. The latest change in surveys has led the authors to throw out highly specific questions and focus on those on which to make decisions and track the effect.

To better serve decisions at the company level, we opted for more generalizable questions focused on five outcome measures and four additional theme areas that are drivers of our outcomes The main outcomes are: satisfaction, productivity, speed, ease, quality. Themes can be viewed in the attached image 8.) Finally, the authors provide an algorithm with recommendations for creating their longitudinal survey program.

• Establish a clear and unique goal for your survey (ensure that there are no other survey programs or data sources that you can already use).
• Collaborate with domain experts and researchers to develop an effective instrument.
• Gather stakeholder buy-in (for example, communicate the value of survey data and partner with teams that can act on your results).
• If you are planning to run a longitudinal survey, invest time in the beginning in setting up the right infrastructure, process, templates, and documentation.
• Maintain the health of your survey program (control survey length and sample strategically).
• Be transparent and accountable. The quality of your insights depends on the feedback provided by your developers, so make it clear why they should spend their time on it.

In general, the article as usual is well organized and quite practical - you can take and use this algorithm in your company, you may not notice the relatively small effects of changes in Google, but you can assess the effect of large changes.

#Management #Leadership #Software #SoftwareDevelopment #Architecture #SoftwareArchitecture #Metrics #Devops #Processes