Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases

The host warns this is an unusual episode: they read an Amazon white paper rather than a book, and get through it in one sitting. By Alexander's reckoning the paper came out "about six years ago, in 2017, I think"; it has eleven parts, but the substance sits in the first eight. Amazon starts from classic relational databases designed thirty or forty years ago: a layer cake of query processing, transactions, logs and storage on a single machine. You can replicate the whole thing or ship log changes, but the storage, cache and log layers still stay on every node. Hence the idea of cutting off the bottom part: distributed systems separate compute, and the bottleneck moves from the disk into the network between database and storage. What gets in the way, per the paper, are synchronous operations: a cache miss going to disk, and a transaction commit. One participant asks the group not to say "sharding" here: this is about replication into segments and protection groups.

One of the guests adds what the paper leaves out: underneath a standard database sits a very large and expensive block store, which makes the database an expensive object you do not earn much from selling — the economics, he suspects, were among the motives. Alexander works through the quorums. In the classic scheme, to survive the loss of one node you take three and make the read and write quorums intersect on a node that knows the latest write. Amazon frames it wider: a region contains availability zones joined by a fast network, and failures inside a zone are correlated: the power goes out and every node in the zone goes with it. The goal is to survive the loss of a whole zone plus one more node, hence a write quorum of four out of six and a read quorum of three out of six. After a zone is lost writes still go through; add a failure on one node and writes stop, but reads continue and no data is lost.

Writes go not to a whole volume but to 10 GB segments, which the fast network rebuilds quickly wherever they are needed. Six segments spread across three availability zones make a protection group, and volumes are assembled from those groups — what used to live inside one disk is now a layer smeared across the block store. The participants underline: the design follows not from theoretical maths but from repair time — how often you fail and how fast you recover. Which yields an unexpected dividend: a hot node is marked as needing repair and the system moves its segments elsewhere, so failure recovery becomes a way of changing configuration and applying operating-system patches, rolled out separately to limit the effect on users. With six nodes you can move one almost with impunity: if a zone drops meanwhile, reads survive and writes come back as soon as the segment has moved.

Vitaly, introduced as the group's chief database specialist, takes the classic picture. At this scale a disk write is network traffic anyway, sometimes over a dedicated network. Two kinds: a small stream of transactional log records, and large writes of table blocks, updated whole. Because the same positions are overwritten, double logging guards against corruption: before a block is overwritten the same block goes into the log, and it is all multiplied by writes to mirror disks. Draw it with local disks and no network, and a crashed database would take so long to recover that nobody would use it. Aurora removes the duplicating writes, cuts the data into gigabyte pieces and lets many replicas run on one storage layer, with a much reduced log stream between them. Reads, Vitaly notes, have not gone anywhere: an evicted page is still fetched from the storage server as a whole eight-kilobyte block, so the fat arrow back remains.

Network storage also costs the operating system's page cache, so prefetch must move explicitly into the database — into index and table scans and the vacuum code. Vitaly takes the write path: a record lands in an in-memory queue, is written to disk, and only then does the node say it has persisted — plus one towards the write quorum. Applying happens in the background: encoded changes are laid onto the previous version of a page, expressed as "add such a record" or "split this B-tree page"; old snapshots are garbage-collected, a scrub validates the data, and anything missing is requested from peers. Hence the phrase that stuck with them, from part three by the speaker's reckoning: the log is the database — materialisation serves not correctness but read latency, and when it falls behind a read takes the nearest snapshot and replays the rest online. Recovery is built from classic ARIES, whose expansion Alexander admits he never memorised: redo after a crash, then rolling back unfinished transactions. In Aurora log replay moves into storage and onto replicas, while undo stays — it matters more for MySQL with its explicit undo files, whereas PostgreSQL keeps old row versions in the table block itself.

Part four is packed with three-letter abbreviations, starting with the LSN the primary assigns to log records; Alexander's "RAM ran out", so he wrote himself a page of expansions and read the rest of the paper with it to hand; he also drew a diagram: a transaction contains mini-transactions, whose last LSN is the consistency point. Vitaly points out that the paper does not invent the LSN — in ordinary databases it is not one, two, three but byte offsets in a virtual file of all the logs. Alexander had expected magic about multi-leader setups and found one increasing sequence and one master. Vitaly adds that Amazon did build a multi-master for the MySQL flavour and then buried it, having decided that the people asking for it wanted something else. Next to it lives the LSN allocation limit — like queue tickets, only so many of which may be printed before service catches up; ten million, if the speaker is not mixing it up. In the benchmarks Aurora is compared with MySQL 5.6: as the node grows it scales almost linearly and MySQL does not. Alexander was amused that the font changes between two neighbouring charts, while on a real workload replica lag drops, by his recollection, from seconds to about five milliseconds.