Distributed Systems — Episode 3

The chat asks which edition they are reading: the latest, the fourth. Before turning to threads, the host recalls the same author's operating systems textbook — his copy was a black brick from the Classics in Computer Science series, its chapter on processes goes deeper, and the distributed systems book keeps referring back to functions an operating system provides inside one machine. One of the guests develops the point: resources already have to be coordinated inside a single machine, between processes and parallel threads, and the same problems recur once you add a network. Then the cost of switching: a process is heavier than a thread, and on an interrupt most of the time goes not on the switch itself but on the indirect effect — after the handler runs, the cache has to be filled again. The book quotes a figure of 80%, though the participants note it all depends on how the experiment was set up.

The authors give four reasons to use threads. Blocking calls: in a single-threaded application one such call stops everything, and the next request waits until the previous one finishes. Efficient use of several cores — but only with careful synchronisation, so that threads do not block each other. An application made of many processes keeps doing context switches, and switching between threads is cheaper. And structuring the application: a separate piece of work can go into a thread rather than a process. Then three implementation options: user level, kernel level and a combination, with one-to-one, many-to-one and mixed models, the last fitting multi-core systems best. The participants laugh that in the comparison table the hybrid option ended up with nothing but advantages. And they voice a complaint: the authors do not call things by their names — had the thing described been called coroutines, reading would have been easier.

Why a book on distributed systems has a chapter on threads at all is explained through clients and servers: many logical connections have to be held at once, and at high traffic blocking calls are no way out. A classic process-based server spawns a separate process for every new client connection to the database, but in practice a thread-based proxy is put in front of it to hold the connections permanently. One participant notes you cannot design a system without knowing what an incoming byte opens — a thread or a process — because connection counts run into hard limits. Alexander recalls Apache, which forked a process per request, and nginx, which arrived in front of it once Apache stopped coping with the load: nginx has a master process and workers, each holding many connections, running across them to advance each request's state machine and ship the client the next chunk from the slow Apache behind it.

The upshot is the familiar two-by-two matrix: parallelism against blocking and non-blocking calls. A single process with blocking calls is easy to program, a state machine with non-blocking calls is hard to program but lets one core serve tens of thousands of requests per second. Alexander's metaphor: a simultaneous chess exhibition on thirty boards, where the grandmaster makes a move and walks on while the state of the game stays beside him; in the process model a new grandmaster would be needed at every board. The chat asks whether a state machine works when the order of operations matters — the order is precisely what the machine defines. Then the four server problems: concurrent or iterative execution; how to find where to connect, through well-known ports, DNS, service discovery such as Consul with its SRV record, or a super-server starting a process on demand; how to interrupt a request — the authors joke the drastic way is to switch the machine off; and stateless against stateful.

Virtualisation is an intermediate layer that unglues a program from the interface of particular hardware. It started with portability: powerful machines came out more often than new software, and old code still had to run somewhere; the cloud gave it a second life, with dependencies packed together, easy movement and replication. A separate value is isolation: a cloud customer has to be promised that nobody will get into their capacity or processes. The participants recall bare-metal servers where a database and an application interfered and there was no telling where the resources went — and admit that noisy neighbours have not gone anywhere: a provider sells four times more cores than it has, and under load you get a quarter of what was promised, less still when throttling kicks in. Process virtualisation means a Java machine, or Wine emulating the Windows API, while a virtual machine monitor runs on a host system that already has the drivers. Containers, by the host's reckoning, were not a settled technology back in the third edition, from 2017, and Mikhail recalls LXC being used at Avito before any shared orchestrator: they read up on cgroups and wrote the resource configs by hand.

The client side goes quickly: a thin client such as VNC pushes the picture pixel by pixel, X Window System is remembered as hours of pain over a non-standard monitor, and Chrome, per the book, is comparable in codebase size to Linux and has effectively become an operating system for applications, with a process and a sandbox per tab. The last topic is code migration: data is normally brought to the process, but code can be brought to the data instead, and the participants point out that MapReduce came from exactly that, with federated learning as a recent case where data cannot be exported. The same move works on devices: a phone used to hand computation to a server, whereas now models are applied and fine-tuned on the device. The book's classification — remote evaluation, code on demand and mobile agents, of which the host honestly says he never grasped the authors' meaning — is completed by weak and strong mobility: weak moves the code and restarts it in the new place, like a pod in the container world, while strong carries the execution segment too.