[2/2] How Dora Reports Are Collected (DevOps Research and Assessment)?
Continue. story As for the DORA methodology, I would like to say that 2024 The authors found a fifth metric of the rework rate, for which the proxy meticulous is the change failure rate. To check, they added a question about the number of unplanned deployments that were needed for fixing bugs. As a result, the hypothesis was confirmed and the researchers supplanted four previous factors and a new fifth in two groups of software delivery: throughput and stability.
- Throughput: change lead time, deployment frequency, failed deployment recovery time
- Stability: change failure rate, rework rate But then the authors decided to keep their past structure with clustering and grouping levels into four categories:)
When it comes to methodology 2024 Years ago, the guys gave 3 individual paths for the survey (where participants were randomly assigned)
- AI (To determine the impact of gen AI) Platform Engineering Impact of Platform Engineering
- Workplace (Influence Transformational Leadership
We then conducted a survey and checked. internal and external validity of measurements: Internal validity showed that answers to different questions about one concept yield consistent results. For this purpose, the authors used confirmatory factor analysis (CFA) applied lavaan R package. External validity showed that the construct collected by the researchers is connected to the real world through interconnections. For example, the authors had expectations for the connection of constructs with each other and some correlation that could be tested in practice. (And if there were contradictions, they required new hypotheses to explain.).
For causal inference, the authors used Directed acyclic graphs. (DAGs) tool DAGittyIt allows you to build a model of the relationships of different factors, as well as try to eliminate the influence of third variables and mimic a / b experiments. A further analysis of the effects of the authors made using n Bayesian approaches
We use Bayesian statistics to calculate a posterior, which tries to capture “the expected frequency that different parameter values will appear.” The “simulation” part is drawing from this posterior more than 1,000 times to explore the values that are most credible for a parameter (mean, beta weight, sigma, intercept, etc.) given our data. Separately, it should be noted that the authors abandoned one large model and made many small models, where they linked capabilities. (conditional engineering) result (outcomes). They then tested the relationship between capabilities and outcomes by trying to separate the effects of third variables. Having done this, they did not just draw conclusions about correlations, but postulated the presence of causal relationships.
In general, the methodology turned out to be definitely interesting, and the next issue of the Code of Leadership podcast will be devoted to its detailed analysis.
#AI #DevOps #DevEx #Metrics #Processes #Management