What Goes Around Comes Around... and Around, Part II

Alexander recalls that part one covered data models and query languages; now they reach the systems architecture section, where the authors show how network throughput, disk IOPS and the cost of compute changed over twenty years, and how those shifts redefined the shape of solutions. He liked this part precisely because you rarely zoom out far enough to see the trends whole. Alexey joined T-Bank as an Oracle administrator, has been there more than seven years and now runs the team behind the company's platform databases; he notes the change of tone — in part one the authors sorted technologies into living and dead quite categorically, whereas here they mostly set out the upsides. He describes the baseline from practice: a table, rows, blocks of eight or sixty-four kilobytes, a pointer to the next block when a row does not fit. You find the address through an index, read the block, and the DBMS returns the whole row and picks the fields itself; the exception is index only access.

The first change against that baseline is columnar storage, with ClickHouse as the bright example: columns live in separate files, so compression is cheaper and vectorisation works far better; Alexander also mentions another fast database that is convenient to run locally for analytics. At their company ClickHouse is already offered as a service in the DBaaS, and Alexey adds a moment of local pride — it came out of Yandex. On DB Engines, whose statistics many treat sceptically, the top is still Oracle, MS SQL and MySQL, but ClickHouse usage keeps growing; alongside columns it brings clustering and a step towards OLAP volumes, while in OLTP the same trick works worse and is usually not needed. Oracle Exadata, says Alexey, moved the same way: the storage node knows how rows and columns are laid out and returns upwards only what the predicates need. Alexander recalls Vertica, which one of the paper's authors, Stonebraker, had a hand in; they ran it too, and it was eventually eaten by ClickHouse on one side and cloud databases on the other.

The second architecture is the cloud database. Alexey states the idea: storage and processing move onto separate nodes, and the processing layer can be stretched dynamically for the daytime peak and shrunk again. Alexander explains what made that possible: slow networks used to force compute to sit next to the data, gluing the two scaling drivers — data volume and processing intensity — together; then object stores such as S3 became the de facto standard and networks became fast. You do not want to shrink data, it is valuable and should simply sit there, but you do want to move compute with the load. His OLTP example is AWS Aurora, which Jeff Bezos named among Amazon's technological breakthroughs in his shareholder letter around 2014: compute was sawn off from storage, only the write-ahead log travels between master and replicas, and the data is pulled from object storage into a memory cache. That lifted the ceiling on reads; writes still hit the single master. Aurora, Alexander recalls, was covered with Vitaly Kondratov and Igor Maslov on their internal Code of Architecture channel.

The third type is the lakehouse, which the authors, in Alexey's retelling, call another solution inflated by cloud platforms: a single infrastructure carrying both SQL and non-SQL workloads over object storage, where data is buried in Parquet or ORC — ORC, he admits, he has never met, internally it is mostly Parquet. Alexander sees the same compute and storage separation applied to a domain once called DWH and then the data lake; he separates ETL from ELT, calls the lake his own purgatory, and laughs at naming in which warehouse, lake and house keep merging. The practical ceiling of the old approach is Greenplum, of which the company is, by their account, one of the largest users in Russia: a cluster of shards on a fairly old Postgres where compute and storage are glued together. The data still fits, the analysts' query load no longer does, and compute cannot be scaled on its own. In the new approach — Parquet, the vendor-neutral Iceberg catalogue and an engine like Trino — the query workers and the data layer stretch independently.

The fourth class is NewSQL; Alexey jokes that imagination did not travel far with the name, and the point is to bring strict transactions back to where they were absent — Mongo, Cassandra and other large stores. Alexander supplies the theory: ACID against BASE, whose acronym he honestly cannot recall in full, and the loss of linearisability, a single order of events that everyone observes identically. NoSQL never had it, and relational databases avoided serializable because it wrecks performance. Eric Brewer stated CAP first as a conjecture, somewhere in the 2000s: under a network partition you choose between consistency and availability. Google hired Brewer — he is now, as far as Alexander remembers, a VP in Google Cloud — and the company built Spanner with the True Time component on atomic clocks; similar databases were later built on NTP instead. Alexey's fly in the ointment: the model rests on mutual locks, and under load everything runs into them.

Alexander agrees that proven solutions win in practice: sharded Postgres is understandable to operate, whereas Spanner, or Yandex Database with its different, Calvin-style deterministic transactions, is a magic box priced like a jewel. Accelerators and blockchain databases the two of them nearly skip: Alexey downloaded a survey of GPU databases from 2024 but did not get to it, and Alexander says he never grasped the idea behind blockchain databases. Then the authors' conclusions. Marketing must not be dismissed: Oracle early in its life, MySQL in the Apache and PHP era, MongoDB in the Mongo, Express, Angular and Node.js stack of the 2010s all rode on it and polished the product after winning the market. Open source from big tech is sometimes published and then abandoned. The first installation experience matters because adoption starts from the bottom: an engineer plays with it, tells colleagues, and bad word of mouth buries the product. The authors' hatred of ORMs did not resonate with Alexey — in vendor software, reporting systems and 1C you cannot scale development without that layer; Alexander recalls Postgres 9 in, he thinks, 2013, where the degradation came not from the ORM but from limit-and-offset pagination. On ML both are restrained: query optimisation at best, though Alexander points to Google's evolutionary optimisation system, which rewrote Borg's scheduling and saved 0.7% of the company's compute.