Why Is My App SLOw? Defining Reliability in Platform Engineering • Jez Humble • GOTO 2023

Interesting. report from Jez HumbleSRE team that supports serverless platform services in Google Cloud. The main problem is the following: within the framework of this platform it is necessary to monitor its performance, but Focus on customers’ perceptions of how well their applications perform on top of the platform You can focus on latency applications, but they have a latency spread of five orders of magnitude. (milliseconds) You can focus on the request delivery metrics, which measure the time from getting into the Google network layer to the delivery of the request to the application and the reverse part of this cycle - but this is a slightly artificial metric.

In the end, the guys set themselves the goal to come up with a metric with the following properties: *- A metric that represents the customer experience

• Combinable across projects / cells / regions
• Can be used to detect anomalies affecting multiple customers (likely platform issues)
• Computationally cheap (high QPS)
• Principle-based*

And they found it by understanding statistics and using the concept of stationarity. (The property of the process does not change its characteristics over time). In short, they identified individual cohorts of queries based on the version of applications deployed in the serverless platform. Further, for each cohort, they checked whether the distribution of its latency fits into the lognormal distribution. If it fits, then the distribution parameters were kept in the base, if not, then this cohort was rejected. As a result, half of the loads fit into the lognormal distribution. Next, the guys use the approach 2 sigma 2 They take the standard deviations from the average for each of the cohorts and collect this in a graph, where you can see what percentage of such workloads with such a large deviation. The normal state of the platform is 2% -5% of workloads with deviations more 2 Sigma, but if this figure rises to 10%, this is a cause for concern. This distribution is usually used for 15 minute window. The author goes on to explain how this works in detail, but I’m interested to talk more about the conclusions that Jez sums up at the very end of the report. *- We can reliably detect and measure the impact of platorm regressions

• Reliability is a shared property (between customer & service) -- Reconstruction of end to end behavior is critical
• Metric combinability is critical for analysis
• Variability is what customers actually care about
• Distributed systems ofen produce decorrelation -- We can measure it, and its absence
• Workload correlation can identify proximate causes*

Overall, this is an interesting report for those who build platforms and want to build SLO/SLA/SLI taking into account not only their technical metrics, but also how user loads feel inside the platform.

P.S. Slides available here.

#SRE #SystemDesign #Software #Architecture #Metrics #SoftwareArchitecture #Engineering #Math #ContinuousDelivery