Continuous Architecture in Practice — Episode 4

The conversation opens with why the authors bet on resilience. By their logic the need has shifted: availability now means keeping at least a minimal set of functions alive under any problem, with Amazon, Google and their peers setting that always-on fashion. One of the hosts notes that he has run into both Google and Amazon being down, and recalls a talk by the team behind the state services portal: banks and large services raised expectations so far that people now expect the same reliability from everything, and the portal aimed at that level, by then a norm. Four-hundreds and five-hundreds used to be a constant sight and are now rare. An announcement follows: early next year their ice-rink event for IT people returns, official announcements still to come, and by the hosts' promise it will beat last year's.

The chapter, like the others, opens with definitions — five in all, among them fault and failure, high availability and resilience. The participants agree the terms are best remembered in English: in translation the Russian words for fault and failure blur together, whereas the distinction is fundamental — a resilient system is built so that a fault does not turn into a failure. That leads to a complaint about literal translations, after which terminology stops being consistent. Zhenya describes how, translating the DDD books, they kept a large table: how a term was rendered in Evans, how in Vernon, how the team renders it and how people actually speak among themselves; they want to publish the glossary, which by his reckoning holds some seventy or eighty core terms. Reliability, the authors say, is the probability of running without failures over a given period in a given environment: it sits on top of availability, and a system can be available yet unreliable.

The main argument of the episode is how the authors split high availability from resilience. The first bucket holds external means: virtualisation, recovery, failing over to a neighbouring data centre, with nothing to change in the application; the second holds what an engineer builds into the code. Sergey Baranov argues that the approach and the techniques implementing it have been mixed up, and that the techniques quoted are ten to fifteen years old rather than today's cloud-native ones; to him high availability reads as a buzzword around which practices later grew, and he has not met such a division in the industry. Worse, the chapter's first half sets the two literally against each other — one or the other — mentioning hybrids only later, although in practice they always worked together. Zhenya adds a familiar mindset: why think about resilience when my database is super-reliable? That was the reasoning ten years ago, before microservices and clouds, when you could walk over to the admins and ask for a super machine.

Alexander moves the argument to the application level: do you treat the building blocks beneath you as magically reliable, or do you build in timeouts and other fault-tolerance patterns, multiplying probabilities across shards? He recommends a cloud provider's document on designing on top of highly available infrastructure: how zones and regions work, how a zonal service differs from a regional and a global one, and what static stability means — during an incident it is better not to call the control plane but to ride out a zone failure on resources already provisioned in the data plane. Alongside it he recalls a paper by two Google authors, an engineering fellow and a vice-president for cloud infrastructure, on cloud deployment archetypes: it covers the case of legacy software deployed whole with a hot standby in a neighbouring zone — precisely the high-availability picture from the book. The question is what that costs in the cloud, but for customers it is about business continuity, where multi-zone is nearly the default and a multi-region fallback brings a non-zero RPO.

The authors then break reliability into four groups of questions: what to do during an incident, where to look, what actually happened — post-mortems, where they advise reviewing the good cases too, since success, the participants note, is faults that never became failures — and what we expect from the system, up to predicting likely problems. One participant jokes that the two eternal questions of Russian literature would have done instead of that foursome, and is told that asking who is to blame is exactly what not to do: the question teaches people to sweep problems under the carpet rather than asking which service did what. The nines take a beating too: the business asks for five nines without understanding what that means, the gap between adjacent nines is sometimes immaterial, and seasonality — holidays and sales — is not in the concept. MTBF, the participants remind, came from failure statistics of microchips, where the hardware is stable while software changes daily, so in practice people aim at MTTR trending to zero; asked for an example, Alexander recalls the Aurora paper, where the authors do not attempt to compute MTBF and lean on ten-gibibyte segments the network can rebuild in under a second.

The catalogue of tactics goes by quickly: health checks, where the Kubernetes probes are the hardest to tell apart, the startup probe among them; alerts that, in excess, turn into an endless saw and stop working; isolating a service onto its own machine; default values as a fallback — serving history without personalisation when personalisation is down; separate connection pools per scenario; a two-level cache, on the principle that some data beats none; backpressure, load shedding and rate limiting, which unlike the others is tied to a client and an agreement rather than to the system's state; timeouts and a three-state circuit breaker; rollbacks versus compensations; and backups, checked for restorability and consistency. The final chapter, on blockchain and artificial intelligence, is shallower still, but the participants pick up its point about the architect's role: dismissing an unfamiliar technology is a sure way to become the organisation's bottleneck, and authority over new topics is better handed to the teams. The verdict: the first two chapters are the strong part, interest fell away after them, the running example did not land and the depth was missing — yet as an overview they would still recommend it. The club signs off until next year, unless an interesting paper turns up sooner.