Building Evolutionary Architectures — Episode 4

The hosts set the frame straight away: the chapter is largely a digest of a 2006 book on refactoring databases written by one of the current book's authors, and Alexander went back to it to compare.

The participants notice something else: every data book they had read before opened with the formula that systems come and die while data stays forever, and here the authors seemingly push back a little for the first time — data ages too, has to evolve too, and part of it belongs in an archive.

The chapter's running theme: developers have tests and refactoring, while the people responsible for data seem to have none; the book keeps separating developers from DBAs with their own world, processes and vendor-tied tooling of the Oracle kind. From the same era comes the migration committee of the most senior DBAs approving big schema changes: almost none of that, the participants note, is left.

Then comes a run through the recommendations: treat data like code, test it, version the schema and pin its version to a version of the code, change it incrementally, use automated tools — in Java, for instance, Flyway is popular, and applying migrations has long lived inside the pipeline.

Undo migrations get their own thread. The authors advise against writing them: you end up maintaining and checking two versions of the schema, and you cannot know in advance how you will want to roll back or whether the data must be kept. The participants add that undo scripts written retroactively fork the migration history, and that an untested undo migration is a parachute nobody has opened. Alexander pushes back: he recently had a major incident where the second step of a two-step migration proved irreversible and part of the data had to be restored afterwards.

The chapter's main example is several applications on top of one database, although the authors had been steering away from that shape earlier. The proposed pattern is expand/contract with a transition period in which two schemas live at once and data sits in both formats.

The participants lay out the variants: a clean database with no integrations or accumulated data exists only during development; with a single integration point the historical data can be converted in one go — in the book that is splitting a name into first and last; and in real life the application runs in several instances, the release goes out without downtime, and old code inevitably meets the new schema.

The book proposes triggers that mirror writes into both formats, removed along with the old column after the transition. One participant objects that a team may move to the next sprint without removing the temporary triggers. Hidden dependencies then remain, so a later change to an apparently obsolete column can affect a working system.

The second half of the chapter is about entanglement and transactions. The authors describe coupling through data as particularly strong, and the chapter compares transactions to the strong nuclear force; two-phase commit is named but barely examined. Nikolai illustrates the compromise in distributed processes through overbooking: a passenger arrives at their seat on the plane and someone is already sitting in it — instead of ACID you get an apology. If the transactional context leaks outside, the service boundaries were probably drawn wrong.

That leads to the question of data ownership: one service controls writes. Others can read through it, keep a cache, or build a local projection from its change stream. The choice depends on the cost of network calls and the need to keep neighbouring applications from changing another service’s data directly.

Fitness functions turn up as well: data has spread across several databases and has to be deleted in each, so the solution is an event that every application listens to and cleans up its own piece, with a fitness function checking the distributed deletion. The participants joke that a fitness function is obliged to appear in every chapter, and note that here it adds coupling between quanta that are meant to be independent.

The retention discussion starts with the habit of keeping everything. Participants describe petabytes in HDFS that turn out, during a migration, to contain only tens or hundreds of useful terabytes. The chapter suggests asking the business what is actually used and moving the rest to cold storage.

Nikolai challenges the idea of an indefinite archive: beyond business value, user agreements, retention periods, and deletion requirements need discussion with lawyers. Cold storage does not remove these questions. The legal case he mentions is not checked during the discussion; the host asks for a link to be added later.

Nikolai’s verdict is that the chapter identifies real problems but proposes solutions he considers dated. A major improvement is JSON support in databases: additional fields can be introduced without a separate schema change. With fifty shards, that avoids having to alter every shard’s schema at the same moment.

The application must still handle a missing field correctly. This does not eliminate data modelling: Nikolai distinguishes fields frequently used in joins. JSON helps with the changing part of a record, but applications still need an agreement about what those fields mean.

Distributed integrity remains harder. The participants discuss how two-phase commit slows as more participants join and why preparation becomes expensive. Sagas offer a different trade-off: a simple main path with compensating actions that restore an acceptable business state, not necessarily the exact starting state.

Someone must still trigger, retry, and check the compensation. In Alexander’s experience, teams will implement that logic for a critical scenario but hesitate to extend it everywhere. This is part of the cost of splitting a system: independent development does not remove the work of coordinating data.

They also discuss NewSQL as an alternative, combining scaling with transactional guarantees. It can look like an ordinary database to its users while remaining complex and costly to operate. Nikolai expects a move towards a single database interface and a larger role for data modelling. His remarks about difficulties Figma might encounter after splitting its database are a prediction, not an observed outcome.

The closing question asks about text search in a JSON column in a 40 TB database. Nikolai says parallelising across hundreds of blocks can produce a fast answer, but running that search regularly is expensive. Occasional analytical queries and continuous workloads need different choices; the answer contains no database comparison or benchmark measurements.