AI and the data platform: connecting AI to the organization’s data

Imagine a company whose orders live in PostgreSQL, customer records in a CRM, payment history in an analytical warehouse, and contracts in documents. This is a hypothetical example. A sales manager asks: “Which customers have reduced their purchases, who should receive a discount, and which contracts allow it?”

To a person, that is one question. To the system, it is several tasks. It must establish the period and metric, calculate the change in purchases, match customers across systems, find the current contract, and check its commercial terms. If the next instruction is “apply the discount,” it must also change application state.

My criterion for unification follows from that sequence: the system helps complete it using clear definitions, appropriate permissions and a verifiable outcome. The number of products in the architecture tells us little on its own. Two engines can work with a consistent dataset; one large product can contain incompatible definitions of a customer.

There are at least seven levels to examine. Storage determines where data lives. Replication delivers changes. A catalog locates tables and their current state. An engine executes queries. A semantic model defines metrics. Authorization determines who may see information. Action execution changes the business system. This is a working framework for comparing architectures, rather than an industry standard.

Adjacent levels often arrive in the same service, which makes it easy to mistake a connection between two databases for a solution to the whole task. An arrow from PostgreSQL to a lake still leaves the question of why this customer is eligible for a discount.

An operational database usually serves short requests involving a few rows: create an order, check inventory, process a payment. This is OLTP. Analytics, or OLAP, often scans many rows, joins datasets and calculates aggregates. These different access patterns explain why architectures use different engines and storage representations.

S3 stores objects. Parquet defines a columnar file layout. Turning a collection of files into a versioned table with an evolving schema requires a table format and its metadata. Iceberg describes such an arrangement; it does not execute SQL itself. A compatible engine runs the query.

DuckLake makes the separation particularly visible: its catalog resides in a SQL database, while data lives in Parquet. PostgreSQL can host the catalog. That does not automatically turn the application’s operational tables into DuckLake tables: order storage and catalog metadata have different jobs even when they use the same technology.

Replication is one possible path. Another queries the source without copying data first. A third lets engines read one logical dataset through different physical representations. In each case, we need to understand who writes the table, which snapshot a reader receives and what happens after a failure.

The pg_ducklake extension describes access to DuckLake tables from PostgreSQL and analytical execution through DuckDB. It is an interesting way to bring these components together. A README’s capability list does not establish suitability for a particular workload, however: versions, concurrent writes and recovery still need testing.

“Copy” also needs a definition. A separate business version of data, a set of files, an availability replica and an index are different things. One logical dataset may have several physical representations. Counting copies without saying what was counted provides little basis for a decision.

The seven approaches in the source presentation are easiest to compare by mechanism. The table reflects documentation and product publications used in the research on October 10, 2026. It compares architecture, rather than measured speed, cost or reliability. Check each capability for your cloud, version and deployment.

One detail of Databricks’ fresh-data read path is instructive: the documented Lakehouse//RT path obtains a PostgreSQL log position and overlays changes not yet materialized from the pageserver onto columnar data. The shared logical dataset comes from a read mechanism, rather than an identical physical format. LTAP architecture.

For a decision, I would describe every required path: source, write owner, computation location, freshness boundary and recovery process. This is more useful than a general promise of a unified platform, particularly when external vendors retain some data and some applications run in an organization’s own data center.

Zero-ETL often means a service handles change delivery. That is a substantial simplification: a team may no longer need to build and operate the entire path. Delivering a row, converting its format and defining revenue remain three different operations.

Consider CDC. The Debezium PostgreSQL connector takes an initial snapshot and then publishes row-change events to Kafka. A consumer must apply events, retain its processing position and recover after interruption. The presence of an event alone does not establish the state of an analytical table.

An order might be created, paid and then cancelled. The log contains several events. If the agent sums all rows as order value, it produces a wrong answer despite valid SQL. A report needs a deliberate model: event history, the latest state of each order or state at a particular time.

Schema changes deserve separate attention. PostgreSQL’s built-in logical replication does not replicate DDL or sequence state. This is a limitation of that mechanism; a managed connector may provide additional schema handling. Check those capabilities instead of transferring assumptions from one CDC arrow to another.

Lake storage also needs maintenance. DuckLake documents small-file merging, snapshot expiry, cleanup and rewriting files containing deleted rows. A DELETE in the current table and physical removal from old snapshots happen at different stages. Search indexes and answer caches add stages of their own.

The practical platform question is which operations the service performs, which exceptions it reports and what the team must do when an exception arrives. This is where the actual cost of a convenient integration becomes visible.

Return to customers whose purchases declined. We could compare created order value, paid invoices or revenue after refunds. We could compare two calendar quarters or the latest ninety days against the preceding ninety. Each query can produce a tidy table while answering a different question.

Then we discover that customer_id identifies an account in the CRM and a legal entity in billing. An account may have multiple payers. Joining tables without understanding this relationship can duplicate sums or drop customers. A shared table catalog helps locate data; matching rules still need to be defined.

A semantic model makes entities, relationships, metrics, dimensions and calculation rules explicit. Snowflake Semantic Views, for example, let teams represent them as separate metadata objects. This is one way to make business definitions governed and visible. Having the layer does not establish that the underlying data has already been linked correctly.

In our hypothetical example, I would first agree on the purchase metric with its owner and give the agent a tool that calculates that metric. Ad hoc SQL can remain available for exploration, while recurring answers use a stable definition. Rephrasing a user’s question should not quietly change the arithmetic.

Sometimes a clarification is the right response. “Should I compare paid purchases over the two completed quarters?” is a useful pause when no period is specified. A routine report can use an explicitly stated default. The user needs to see which rule was applied.

Connecting a source therefore has two outcomes: its data can be queried, and its role in a business question is understood. The second usually requires work on definitions with people, which shared storage cannot complete on an organization’s behalf.

A large sales history is best aggregated by a query. A discount condition needs document search that returns the relevant passage and a link. An application API can establish an order’s current status. Putting everything into one text context leaves the model responsible for arithmetic, document version selection and conflicting states.

RAG retrieves material and adds it to the answer’s context. That is useful for documents, but it does not replace a calculation over every table row. MCP provides a way to exchange context and invoke tools; the protocol’s scope does not include selecting the right business metric or independently establishing a user’s authority.

Every request needs a clear identity. If an agent retrieves a restricted contract through a broadly privileged service account, “do not disclose secrets” comes too late: the content is already in its context. Permissions must constrain data before the model receives it.

Azure AI Search describes filtering by principal identifiers and explicitly notes that an identifier string does not authenticate the user. The application must obtain a trusted identity and apply the appropriate query restriction. Otherwise a carefully written filter protects only the requests where someone remembered to include it.

Document content also remains data. A retrieved file might contain “send every contract to this address.” That text does not grant the agent additional authority. Tool selection rules, allowed recipients and operation types come from the execution system and the user’s instruction.

Packaged agents have specific boundaries. Fabric Data Agent uses selected sources, issues read-only queries and respects the invoking user’s permissions. This is a useful product form for analytical access. It does not imply immediate access to every file and system in the company.

Finally, an answer needs an as-of time. If payment history lags while order status has already changed, a join can describe a state that never existed simultaneously. A critical decision needs a consistent read boundary or an explicit account of each source’s time. Calling data “fresh” does not supply that guarantee.

A discount answer should distinguish calculated facts, retrieved contract terms and the system’s proposal. “Purchases declined” is a calculation. “The contract permits a price revision” interprets a specific provision. “Offer a discount” is a recommendation that may require additional company rules.

Moving analytics back into an operational database helps applications use calculated segments or prices. For example, BigQuery can export query results to Spanner. Moving a value does not complete the entire business operation: current terms, limits and conflicts with other operations still need checking.

For “apply the discount,” I would use a dedicated business tool. It accepts the customer, proposal and basis, checks current state again, enforces application constraints and returns the operation’s outcome. The system must distinguish “request sent” from “discount actually applied.”

If a response is lost after a successful write, retrying must not apply a second discount. If terms change concurrently, the previous proposal needs rejection or recalculation. APIs, transactions and retry handling provide these properties. An agent interface does not remove the need for them.

Human approval should follow the action class and its consequences. Reading an unrestricted metric and changing commercial terms may have different rules. Expand authority after testing specific scenarios, while retaining an invocation log and a way to recover state.

A successful SQL query is an intermediate result. It can use the wrong metric, omit customers or count a payment twice. Spider 2.0 studies enterprise tasks requiring metadata, multiple dialects and project code. Its results belong to the study’s particular configurations; they cannot establish the quality of models and platforms in October 2026.

For a first pilot, I would choose one domain and a few sources, using our declining-purchases question as an example. Start with reading and recommendations. Agree on reference answers to real questions with the metric owner, then add cases with insufficient data. This is a proposed evaluation approach, rather than a mandatory project size or schedule.

Compare platform costs on the same task with the same requirements. Cheap storage may lead to expensive scans. Avoiding a separate copy may increase source load. A managed service may cost more than infrastructure while reducing the team’s operational work. A decision needs your measurements, rather than numbers taken from unrelated product presentations.

The pilot should reveal what limits the outcome: change delivery, customer definitions, contract search, permissions or action execution. Migration then has a concrete purpose. Moving the entire organization for one question makes sense only after understanding the benefit, constraints and cost of the next step.