What Goes Around Comes Around... and Around, Part I

The guest introduces himself: he has worked at T-Bank for seven years, joined on classic relational Oracle solutions, and before that spent five to seven years working for the American market. His team now owns platform and non-platform databases inside the company: support, platform development, and the user scenarios that let teams run workloads on its own infrastructure. Alexander explains the choice of guest: several engines are already offered as a service, and the team keeps adding platform options such as vector databases, so no team has to deploy one itself. The paper is a year old: seventeen pages, thirteen of main text and four of references, and Alexander likes such surveys for the helicopter view and the trends. It continues a series started twenty years ago by "What Goes Around Comes Around" — from 2005, by Alexander's reckoning — co-authored by Michael Stonebraker, a serial database creator: Ingres, then PostgreSQL, then the work that turned into Vertica.

The second author, Andrew Pavlo, is a Carnegie Mellon researcher known for ideas about autonomous databases that take over part of the DBA's job; he came to HighLoad some five years ago, but Alexander missed the talk and watched the recording later — autonomy was hard to believe in then, and today's trends point that way. What Alexey missed in the paper is geography: PostgreSQL evolves differently inside the country than in the global community, and that changes the landscape; Alexander agrees the authors care more about the American and global corporate market. The relational model is absent from the list: it is the de facto standard, the yardstick for everything else. Alexey sees a trap in that — comparison pulls everything back towards relational — while admitting business is tabular by nature. The first paper, Alexander recalls, predicted relational databases would extend their functionality and competitors would not take a meaningful share of the market.

The walk follows the paper's blocks, the first being MapReduce: Google popularised it to spread processing across many nodes and reduce the results. An open-source tool with the same ideas followed — Hadoop — with databases such as Hive on top to hand the familiar SQL back. The model proved expensive: real time is hard to reach, it eats resources, Google itself largely buried its own solutions, and Alexey adds it was pushed into plenty of business systems where living with it proved costly. The authors file it under dying legacy, but Alexander explains why it survives: if the data already sits in HDFS and speed and cost are acceptable, a migration cannot be justified; the company's own Hadoop is no longer its target platform. On key-value, Alexey thinks the authors stopped at fast access by key and a nod to caching, while the case that matters in his market is an external sharding router; Alexander spells the scenario out: the shard stays ordinary PostgreSQL, and the router has to be fast and fault-tolerant, ideally in memory and probably persistent. Redis, he adds, does far more than plain key-value, despite its licensing problem and forks.

Document databases, Alexander notes, are talked about mostly by application developers: MongoDB removed the constant translation of objects into relational structures and back — serialise the object into a document and store it. The proposition worked even without consistency guarantees across collections, and in the early versions there was no real sync to disk, so data could be lost. Later, near-enterprise solutions settled on the engine, transactions had to be added, and Mongo became simply a big, fast, sharded database. Alexey looks from operations: JSON beat XML on simplicity, the move from the relational world is easy, but ACID has to be stretched over it, add-ons extend a core never designed for them, and engineers do not enjoy supporting Mongo. The two then work through the real price, the schema: a relational database validates a write at the door and lets you roll migrations, while careless document work pushes the schema into the application, which accumulates a zoo of versions of one document type — much as mobile development drags along old operating systems.

Wide-column is discussed through Cassandra, which the company runs as a service: Google had the Big Table paper and colleagues at Meta wanted something similar. Modelling is inverted — you start not from normalisation but from how the application will write and, above all, read: you pick a sharding key so a query does not scatter across shards, a sort order matching the data layout, and sometimes duplicate it into several representations. Taking a cluster into support follows a procedure: engineers validate the structure with the team, since moving from the relational world produces design mistakes. The pluses are fast inserts and a TTL that cleans records up by itself; the storage engine differs too — instead of a B-tree, a log, in-memory tables and periodically compacted files on disk. For full-text search Alexander advises starting with what PostgreSQL offers rather than deploying a separate engine. Alexey has a story: at a previous job a client's text index in Oracle stopped refreshing, they spent four months on it with the vendor's engineers, and he found a sixth, undocumented view holding rows that failed to sync — the patch arrived two months later.

Alexander explains vector databases from first principles: reducing dimensionality predates LLMs, and a transformer turns its input into a vector as its first step — of dimension 768, say. Writing it down is easy; searching needs a metric such as cosine similarity and approximate nearest-neighbour methods. In exchange the semantics work — ask for "king", get "queen" — and pairing full-text with vector search gives the tool for retrieval-augmented generation. The company tested the hypothesis that it needs many vector databases: two plugins for the PostgreSQL it already runs everywhere — one pgvector, the other Alexey could not name — plus two specialised engines, and, if Alexey remembers right, Elasticsearch and Cassandra on top. Specialised systems win on their storage model, yet the extensions are digestible at small volumes. On array databases Alexander admits he knows little: the idea is a multi-dimensional value in place of a scalar one, for scientific data. He played with Neo4j and praises Cypher, where graph pattern matching alternates with tabular selects; the difficulty is operational — a graph is hard to cut across machines, and traversal depth has to be capped separately: the richer the model, the harder to make it work. The second half, on system architecture, goes to a separate episode.