Databricks about production AI: why the model is chosen near the end of the project (Category AI4SDLC)

Viewed the report by Sandipan Bhaumik of DatabricksThe Production AI Playbook: Deploying Agents at Enterprise Scale", in which the author shared his approach to the introduction of agents in client enterprise projects, where it was necessary to go from the wow effect on demo to production:)

Ironically, most AI projects started with the question “GPT or Claude?” Next, a beautiful prototype was quickly assembled on pure data, the management was happy, and after a few weeks in production no one understood why the agent answers strangely, who is responsible for this and where to look for the cause. In order not to get into this situation, he offers his playbook of five supports.

1️⃣ Evaluation (Define success before code) Fdnjh calls them the AI system specification: Before code and before model selection, you need to understand what counts as success in numbers. Not just “accuracy”, but what accuracy is sufficient for a particular business, how many simple requests should be closed by an agent, what errors are permissible, where a person is needed.

2️⃣ Observability (see everything, always) If the agent refused the client, incorrectly called the tool or went to the same database three times, this should be seen in the steps: intention classification, API call, document search, reasoning, guardrail check, final response. Otherwise, the incident is left to guess.

3️⃣ Data foundation (question + tracking data) The author divides this item into data from which the agent responds to the user, and tracking data - traces of the agent's work. People forgive bad data, agents don’t. The person will see the strangeness in the report and ask again. The agent will confidently give the wrong answer.

4️⃣ Orchestration (patterns that scale) The work of one agent can still be kept in mind. Five collaborative agents already require patterns. Orchestrator-worker, where the central agent distributes the work Choreography, where agents listen to events and work in parallel Human-in-the-loop, where a person enters a circuit with low confidence or risky action

5️⃣ Governance (what keeps you in production) The report is not just about data governance. (which should already be a pre-requisite)Generally speaking, risk and liability management. This is a log audit, checks of personal information, versioning of prompts, the process of changing models in the product and answers to questions such as "who answers if the agent broke down in the market". 3 in the morning.

In the report itself, the author shares a case of a chatbot for a retail bank, where a Databricks client spent about 85 thousand pounds 6 For months on PoC, which didn't come out in production, no one could explain why the system was running at low quality. In the new approach, the team chose a model only for 7- Weekly 8- weekly project. Before that, they gathered around 200 Real cases of people’s responses in the chatbot, described metrics, built a pieplan for evaluation, turned on tracing and prepared a data layer.

Six weeks after launch, a normal production incident occurred: the bank updated its interest rate policy, customers began to give poor ratings to chatbot responses, as the agent continued to refer to an outdated document. Without tracing, it would look like "AI's Behaving Strange Again." With the traces, it became clear that the new document on interest rates did not get into the vector base and the chatbot did not know about it.

From here grows playbook to work with incidents - Detect - see the drawdown through the dashboard evals (offline metrics) or online metrics

• Diagnose Disassemble the tracks and understand what goes wrong
• Contain Take action: roll back prompt, turn the stream to a person or turn on circuit breaker
• Fix Correct data, tool call, prompt or model
• Prevent In the final, be sure to add a new case to the evaluation set so that the system catches it next time.

Databricks, understandably, shows this through its products: MLflow for tracing/evals/monitoring, Unity Catalog for rights and context, Agent Bricks and Unity AI Gateway for managing agents, models, tools and accesses. But the value of the report, I think, is broader than a specific platform.

I like this report because it fits well into the subject that I have. told Just yesterday on Highload++, where AI starts to work in companies well, if it changes the production system. But if there is no harness, observability, data and governance around the agent, then scaling quickly takes you from a beautiful demo to problems on the market.

#AI #AI4SDLC #Engineering #Agents #Architecture #Management