AI and the Data Platform

Alexander Polomodov and Nikolay Golov begin with a concrete problem: connecting AI to an organization whose data is already spread across many databases. MCP access alone does not explain how to reconcile information from different systems. If the combined dataset fits in none of them, sequentially fetching a few results is insufficient. The platform needs somewhere to execute a combined calculation and return a useful answer.

Golov proposes comparing architectural approaches rather than prescribing a single correct design. A warehouse engineer naturally places an agent on curated data marts, yet major suppliers are exploring other routes. The distinction is where unification happens: in a change stream, a query engine, one database, or shared storage. Actual products combine these ideas. The framework therefore identifies trade-offs instead of producing a definitive ranking of vendors.

The first approach brings source data through CDC or ETL and transforms it into analytical data marts. CDC captures table changes; ETL extracts, transforms, and loads data into a destination. Continuous streaming is not always necessary: periodically copying a whole table can be sufficient.

Traditional warehouses have raw, detailed, and application-oriented layers. A medallion architecture describes a comparable progression through bronze, silver, and gold. Curated marts suit agents because their structure and meaning are documented and many inconsistencies have already been resolved. Each transition, however, introduces work and latency. Golov describes projects lasting months or years and processing chains that can deliver information too late for an operational action. These are observations from his experience, not a universal project schedule.

The second approach queries multiple databases through a common engine. Presto and Trino illustrate the idea: the engine knows where tables live, parses SQL, contacts each source, and combines their results. It offers an appealing shortcut to cross-system access without first building a separate warehouse. Small sequential reads can work well in this model.

Large joins expose the difficulty. Substantial data must move between systems, sources respond at different speeds, and one failed branch can affect the entire query. Restricting workloads helps, but reduces the universality of the interface. The participants discuss preserving familiar SQL for the agent while addressing expensive cases inside the platform. This moves the complexity to the designer, who must make explicit performance and reliability trade-offs. Giving an agent a long list of exceptional query rules is itself an unreliable interface.

Microsoft and AWS illustrate a managed evolution of the first approach: the supplier handles the movement of changes into analytical storage. The discussion covers OneLake on the Microsoft side and zero-ETL and Iceberg tables on the AWS side. Avoiding a separate hand-built integration for every source simplifies adoption. It does not remove questions about lag, supported sources, or recovering missing changes after a database failover.

ClickHouse and Google are discussed as hybrids of federation and replication. In the ClickHouse example, an agent can work with PostgreSQL and the analytical engine, while large datasets are copied ahead of time to the place best suited to joining them. Google connects operational systems such as Spanner with BigQuery using change capture and movement in the reverse direction. Choosing which tables to copy, where to calculate, and how much delay to accept remains an architectural responsibility.

Golov also emphasizes cloud processing costs. Extra copies and data movement affect the bill, and a poorly constrained query can perform enormous amounts of useless work. His interpretation of supplier incentives is his own assessment. The practical lesson is to understand the billing model and workload boundaries independently. A convenient integration is not evidence that its cost will suit the organization, particularly when agents can issue queries repeatedly.

The third approach promises transactions and analytics in one system. Row-oriented representations serve operational queries; column-oriented representations serve analysis. The database coordinates them and can add specialized indexes. Oracle, SAP HANA, SingleStore, and TiDB appear in the discussion. This family of ideas is commonly associated with HTAP, or hybrid transactional and analytical processing.

The participants treat the promise as ambitious. Supporting different query types does not establish equal efficiency for every workload. Golov discusses architectural assumptions, price, and supplier dependence. Polomodov recalls teams trying to preserve a way back. An integrated product must therefore be assessed for expansion and exit conditions as well as consistency. The episode does not dismiss the approach, but ties its convenience to accepting the constraints of a particular platform. Moving responsibility inside a database changes who operates the complexity; it does not make the underlying tensions disappear.

The fourth approach uses object storage and an open table format with multiple engines above it. Analytical and transactional engines specialize in their respective workloads while tables remain accessible in a common layer. Golov distinguishes this from CDC: a stream of events still has to be collapsed into current table state, whereas shared storage should expose that state directly. An analytical system using S3 underneath is not automatically an implementation of this design.

Databricks and Snowflake illustrate movement toward connecting analytics and PostgreSQL. The Databricks discussion includes Neon, Mooncake, and a proposed workflow: an agent discovers a pattern, creates a service with an operational database, and lets other agents analyze its data. Golov explicitly describes components still being assembled. Snowflake appears through its acquisition of Crunchy Data and pg_lake, connecting PostgreSQL with Iceberg data. These examples describe supplier directions rather than an already complete universal platform.

Polomodov finds common storage attractive but questions efficiency: operational queries benefit from row-oriented access, while analytics reads selected columns. Golov explains that frequent small Iceberg writes generate many files and metadata operations. Latency, transactions, and object-storage limits remain significant. A shared location can become a shared bottleneck, particularly when a self-operated S3-compatible service must sustain reads, writes, and metadata traffic.

The closing comparison returns to the four approaches. Change capture is a practical starting point, followed by work to make the resulting data useful. Federation is particularly appealing with few systems and a bounded workload. A universal database moves responsibility into a product. Shared storage offers more freedom of computation but requires difficult work on writes and coordination between engines. No approach has established itself as the inevitable winner.

Asked about building a platform independently, Golov says it is possible while cautioning against assembling the entire stack at once. An open format narrows the problem and supports replacing tools, but does not make every implementation above it mature. His reservations about operating DuckLake are presented as his judgment about deployment risk. A dedicated review of open-source options is left for a future conversation. The useful decision framework is to locate the concentration of complexity, test it against the intended workload, and preserve the ability to change course.