Preface and Introduction

In traditional ML, a team collected and labelled data, selected an architecture, trained a model, and then integrated it into a product. Foundation models already carry knowledge from large-scale pre-training and are exposed through APIs, so an AI engineer can start with a user workflow. Text, images, and audio become tokens; the model predicts a continuation from context, while self-supervised learning extracts examples from the corpus itself. This lowers the entry barrier without removing dependence on represented languages and domains.

Commoditisation makes previously expensive tasks practical. The conversation covers medical-content review, moderation, sentiment analysis, code and architecture review, translation, and interface generation. A multimodal model can turn a Figma screenshot into an internal DSL description or match a visual to an article; one general LLM can replace several narrow classifiers. Yet an accessible prototype is not an operational system: quality, safety, privacy, and organisation-specific context still have to be engineered around the model.

The hosts contrast market hype with practical adoption. Business stakeholders expect magical acceleration and ask for AI everywhere, while durable value appears only when a model capability is embedded in a specific job. Evgeny describes an assistant connected to Google Drive, Confluence, and engineering metrics; it locates ADRs, surfaces contradictions, and increases review capacity. Alexander frames this as platform engineering: assist employees inside established workflows, then gradually delegate selected jobs to agents.

Internal applications are often the safer proving ground. The company knows its users, receives fast feedback, and can measure time saved without immediately exposing customers to uncertain answers. Medicine, finance, and other regulated domains have higher error costs and stricter data requirements, demanding vetted vendors or models inside the security boundary. Teams must combine deterministic signals with LLM-based assessment, decompose complex reviews, and retain human judgment wherever the model cannot reliably support its conclusion.

Model reasoning gets cheaper while instruction-following improves, so a marginal use case may become viable within months. Not every operation needs the strongest model: simple requests can go to a cheap or local model, while difficult ones reach a cloud model or stronger critic. Eval-driven development makes this a trade-off among quality, cost, and latency. Probabilistic output calls for ranges and repeatable evaluation rather than one exact result.

The stack is separating into model development, infrastructure, platform services, and AI applications. Most engineers will work at the application layer without knowing every training detail, much as Kubernetes users need not understand all its internals. Rapid prototyping democratises hypothesis testing, but production remains expensive: it requires application-specific evaluations, constraints, and data. Public benchmarks can remove obvious losers, yet their scores depend on methodology; the final choice must be validated against the product's own tasks and operating conditions.