LectureHSE · May 23, 2026
Data Platform Basics
A basic introduction: why data platforms exist, how ETL/ELT loops work, and how maturity is measured
/ Data Platform Basics · HSE 2026
Slide contents
1. Data Platform Basics
A basic introduction: why data platforms exist, how ETL/ELT loops work, and how maturity is measured
2. Alexander Polomodov
Technical Director & Fellow, T-Technologies
Architecture and engineering R&D.
AI adoption in SDLC.
Focus: data platform as product.
3. 01. Why a data platform exists
From raw events to reliable decisions, products and AI scenarios
4. Data becomes a production loop
A platform is needed when data must work repeatably, not heroically
Speed — Source → mart/API/model/report.
Trust — Ownership, quality, lineage, access and freshness.
Scale — One loop serves analytics, product decisions, ML and AI scenarios.
5. Stores and Scripts Break
Accountability breaks
Teams rediscover sources and access.
Business logic hides in unowned SQL.
Data incidents are noticed too late.
Cost is not tied to value.
6. The platform connects data to decisions
It is not one database; it is a repeatable production line for data
7. 02. ETL, ELT and hybrid loop
How data moves from source to consumer
8. ETL vs ELT
Difference: where data is shaped
ETL first
Extract from sources.
Clean and validate before loading.
Strict rules, predictable schema.
ELT first
Load into storage/lakehouse.
Build marts on raw data.
Fast ingestion, flexibility, replay.
9. ETL, ELT and the hybrid lakehouse loop
No single choice wins everywhere: freshness, control and cost define the scenario
10. Reference data platform architecture
Sources → ingestion → Bronze/Silver/Gold → consumption
11. Pipelines Must Be Replayable
The result must be provable
Exactly-once is rarely realistic.
Backfill needs a separate path.
Schemas version through contracts.
Every pipeline has owner, runbook and SLO.
12. 03. Platform operating model
Who owns the platform, data products and quality
13. A data platform is also org design
Technology will not work without clear accountability
Centralized — One team sets tools and rules.
Hybrid — Platform owns capabilities; domains own products and SLAs.
Federated — Data Mesh: domains + shared standard.
14. Central → hybrid → federated model
The larger the scale, the more important it is to separate platform capabilities from data ownership
15. Data Mesh needs a platform
Federation needs a shared layer
Domain ownership
Meaning, rules, priorities.
Data products and consumers.
Freshness, completeness, compatibility.
Platform ownership
Self-service ingestion, storage, compute, orchestration.
Catalog, lineage, access, quality.
Financial and engineering guardrails.
16. 04. Governance and reliability
Contracts, lineage, quality, observability and cost
17. Basic Governance Loop
Insurance against invisible failures
Data contracts: schema, meaning, changes.
Catalog/lineage: where, origin, consumers.
Quality: freshness, completeness, uniqueness.
Observability: degradation, delays, cost, SLA.
18. Where data platforms usually break
Accountability breaks before tooling does
One giant DAG — Local change becomes systemic risk.
Hidden logic — SQL rules lack tests, versions and review.
No economics — Mart and storage cost stays invisible.
19. Data Service Level Must Be Explicit
Consumers see trust level
Freshness: speed after source event.
Completeness: passed expected records.
Correctness: required quality rules.
Incident response: owner, runbook, degradation.
20. 05. Platform maturity metrics
How to know the platform became a product, not a tool collection
21. Data platform maturity metrics
Speed, reliability, reuse and economics should be visible in numbers
22. First 120 Days of Maturity
Start with map, owners, change rules
0-30: sources, consumers, owners, incidents.
30-60: contracts, catalog, quality, freshness.
60-120: templates, runbooks, showback.
Then: metrics → product value.
23. What to take away
A data platform is a managed path from source to decision
ETL/ELT/hybrid answer different trade-offs.
Operating model matters as much as stack.
Maturity = speed, quality, freshness, economics.
Next: DataOps/MLOps in production.
Next: DataOps/MLOps
24. Links and materials
Reference sources
System Design Space: https://system-design.space/chapter/data-pipeline-etl-elt-architecture/.
System Design Space: https://system-design.space/chapter/data-platforms-2025-film.
OpenLineage, Apache Airflow and dbt documentation.
Data Mesh and operating model materials.