Mixed observability for GenAI
OTel substrate + GenAI layer
Slide contents
1. Mixed observability for GenAI
OTel substrate + GenAI layer
2. What this article covers
Scheme → SOTA stack
Parts 1-2: scheme, architecture.
Parts 3-4: GenAI traces, OTel/Langfuse.
Parts 5-6: market and maturity gaps.
Part 7: SOTA stack and verdict.
3. 01. Two-layer observability scheme
OTel as the nervous system + specialized GenAI layer on top
4. OTel substrate + GenAI layer
5. 02. Reference architecture
What the chain from user request to AI backend looks like
6. From request to backend
7. OTel semantic conventions for GenAI
8. 03. GenAI trace best practices
Decision episode: from user request to final response
9. Structure of a good trace
10. 04. OTel/Langfuse: how they coexist
OTel-first, Langfuse-as-AI-backend — the most mature pattern
11. Two integration models
12. Langfuse over raw OTel
13. 05. Four market approaches
OTel-first, APM-first, AI-platform-first, Auto-instrumentation
14. 4 architectural approaches
15. 06. Maturity assessment: 7 pain points
Honest assessment — what's covered, what's partial, what still hurts
16. What already works well
17. Token/cost and quality
18. What still hurts
19. 07. SOTA stack and verdict
Five principles for the right stack for a mixed prod system
20. The right SOTA stack
21. Verdict
The industry matured, but not fully
SOTA = OTel protocol/context + AI interpretation/evals.
End-to-end mixed observability is more standard.
Quality and privacy/governance remain open zones.
tellmeabout.tech · ai4sdlc-research.space
22. References and Materials
OTel, GenAI telemetry, evals
OpenTelemetry documentation and GenAI semantic conventions.
Langfuse, MLflow, Phoenix and OpenInference tracing/evaluation docs.
Datadog LLM Observability; Azure AI Foundry.
23. Thank you!
Mixed Observability for GenAI
Materials and links are in the "Book Cube" channel
Alexander Polomodov, Technical Director & Fellow, T-Technologies
@Book_Cube