LectureHSE · May 23, 2026
AIOps / LLMOps for AI Products
Executive decisions, economics and FinOps for the production AI loop
/ AIOps / LLMOps for AI Products · HSE 2026
Slide contents
1. AIOps / LLMOps for AI Products
Executive decisions, economics and FinOps for the production AI loop
2. Alexander Polomodov
Technical Director & Fellow, T-Technologies
Architecture and engineering practices
AI adoption at development scale
Focus: managed production AI
3. The platform exists. Now manage it
Executives choose behavior, launch path and control
Vocabulary is known — Gateway, context, evals and guardrails.
Focus on decisions — SLA, autonomy and quality cost.
Money anchor — Support bot tests TCO/FinOps.
4. Basic components are assumed
AIOps / LLMOps starts after the platform appears
Skip the platform map
Choose service by scenario
Count solved-task cost
Focus: AI product manageability
5. 01. From basics to governance
Do not repeat the AI platform map; translate it into executive decisions
6. After launch, AI becomes a service
Owners, budget and control loop appear
Quality: scenarios and thresholds
Risk: rights, audit, fallback
Cost: not only tokens
Ownership: product, platform, risk
7. The manager chooses the frame
One model can be assistant, service or agent
What should happen
Which scenario improves
Which baseline it beats
What must not break
Error cost and data risk
Cost ceiling per operation
8. AIOps is a decision loop
Before: scenario and ceiling
During: autonomy and fallback
After: economics and routing
The operating model matters more than any single model
9. 02. Manager decisions
Scenario, autonomy, error cost, SLA, quality budget and ownership
10. Scaling needs decisions first
Otherwise AI grows faster than control
Value — Value and scenario — Value, baseline and SLA.
Risk — Risk and control — Allowed autonomy and human-in-the-loop.
Cost — Economics and ownership — Quality cost; product, data, risk and bill owners.
11. Error cost defines autonomy
Higher consequence means stricter control
Low error cost
Drafts, hints, search
Fast models and cache
Metric: speed and deflection
High error cost
Money, contracts, prod
Policy gate, approval, audit
Metric: incidents and escalations
12. AI SLA is broader than API uptime
Users need a safe result
Service health
Latency, timeouts, fallback
Peaks, queues, rate limits
SLO includes fallback path
Quality health
Sources, completeness, policy
No context means escalation
Measure quality by scenario
13. Evaluation starts from product
The same stack can be right or wrong
Value: process and baseline
Risk: error and approval
Economics: unit cost
Operability: release and rollback
Explain the decision, not only the stack
14. 03. Options and trade-offs
API-first, managed platform, rented GPU, owned self-host and hybrid
15. AI runtime is a portfolio
Choice follows data, control, horizon
Buy more ready-made
API-first: fast launch
Managed: control and lock-in
SaaS: demand validation
Take more control
Rented GPU: OPEX over CAPEX
Owned self-host: control and ops
Hybrid: routes by scenario
16. Assistant and agent are different products
One model, different risk and governance
Read-only assistant
Value: search and explanation
Risk: error and leakage
Control: ACL, citations, fallback
Action agent
Value: executes steps
Risk: money and prod
17. You need routing rules
Path follows scenario, risk and cost
Low risk uses cheap routes.
Expensive errors need premium/review.
Failure means partial/read-only/handoff.
Cost growth changes routing.
18. Choice is a portfolio decision
API-first validates the launch
Rented GPU gives transitional control
Owned self-host needs stable load
Hybrid fits different scenarios
Runtime cannot be chosen without scenarios and financial horizon
19. 04. Money case
Support bot: load, tokens, GPUs, team and the management decision
20. Calculate the support bot
200,000 monthly dialogs · 10 messages · 5-second SLA
What we calculate — Load, tokens, peak, GPU, CAPEX/OPEX, team.
What we compare — API-first, rented GPU, own self-host, hybrid.
What we decide — Year-one path and review metrics.
21. Case parameters set the decision scale
This is no longer a toy demo, but still not a load that automatically justifies owned hardware
22. Translate parameters into RPS and parallelism
1,000,000 LLM answers per month create a peak of about 9-10 concurrent generations for 5 sec SLA
23. GPT-5.2 API gives a clear starting OPEX
With 1,400 input and 180 output tokens per answer, the LLM part costs about $4,970 per month
24. API-first buys speed
The risk is silent unit-economics growth
Why launch is convenient
Near-zero CAPEX
Faster production launch
Thinner surrounding platform
What must be controlled
Output, cache, retries
Provider limits and degradation
25. Size self-host from SLA and peaks
For a peak near 10 concurrent answers, the estimate gives 6-8 A100 GPUs for 14B and about 12 GPUs for 32B
26. GPU rental trades CAPEX for OPEX
In the example, 6-8 A100 GPUs cost 1.17-1.56M RUB/month before network, storage and team
27. Self-host changes the problem
Tokens become an infrastructure product
Rented GPU
Tests the self-host hypothesis
OPEX depends on utilization
Needs MLOps/SRE
Owned hardware
Maximum control
CAPEX and delivery lead time
Makes sense with stable load
28. Strategy comparison shows the year-one decision
For the given load, API-first is usually rational unless there is a strict on-prem requirement
29. The decision must include a review point
Launching via API does not remove the need to recalculate under traffic, compliance or quality-cost growth
30. The case decision
Launch API-first unless owned perimeter is mandatory.
Track unit economics immediately.
Set review triggers.
Self-host must pass calculation, SLA and operations readiness.
The first decision is speed with control, not eternal architecture
31. 05. Manageability and FinOps
Dashboard, cost allocation, unit economics review and a 120-day plan
32. FinOps starts with bill ownership
Product-linked spend grows slower
Each scenario has a limit.
Cost is visible by route.
Shared buckets hide choices.
Price explains the trade-off.
33. Owned hardware makes the bill platform-wide
The Habr/TCO example shows the order: one configuration around 202M RUB, prod/dev/test/reserve around 606M RUB
34. After CAPEX, operations begin
Power, cooling, licenses, support and team turn hardware into a service
35. Shared platforms need clear allocation drivers
GPU hours, storage, kWh, traffic, containers, models and users connect costs to product behavior
36. The dashboard connects quality, risk and money
Separate token charts do not create a decision
Quality and risk
Quality by scenario
Fallback and policy violations
Incidents with evidence pack
Economics and control
Cost per outcome
Cache, retries, route mix
37. 120 days: scenario to scale
Start with a management loop
0-30: scenarios, baseline, owners
30-60: observability, fallback, dashboard
60-90: limited release, unit economics
90-120: FinOps and scaling
Scale managed scenarios
38. No measurement, no management
Without evals, quality looks higher.
Without fallback, risk grows.
Without unit economics, economics break.
Without ownership, disputes begin.
Managed AI starts with manager decisions
39. References
AIOps, risk and economics
OpenAI pricing and docs.
OWASP, NIST, Langfuse.
Habr TCO, FinOps Foundation.
40. Feedback form for day two
Feedback form for day two