Skip to content
back to the archive page
#AI

Code of Leadership S2E10: PRD == Evals: How AI Erases the Boundary Between Product Manager and ML Engineer (Category AI)

#AI #Product #MachineLearning #Evals #Engineering #Agents

This Wednesday at 17:00, Albina Munirova from T-Bank and I will discuss in a live stream the latest changes in the product manager profession as the boundary between product manager and ML engineer becomes noticeably thinner. A prototype can now be assembled in a few days, but the main question begins after the demo: who will turn product intent into reproducible checks and take responsibility for the behavior of a probabilistic system?

Albina Munirova is a product lead at T-Bank's AI Center, a former ML engineer and team lead, and the lead and creator of the ML product manager role at the bank. The central formula of the conversation is PRD == evals. We will unpack what it means in real work and why it is not simply replacing one document with a table of test questions. Albina also has her own YouTube channel.

We will discuss:

  • Who an AI Product Builder is, and whether this is really a new profession;
  • What a product manager must now be able to do hands-on, and where MLE specialization remains critical;
  • How to translate requirements for an assistant into an eval set, a release threshold, and product metrics;
  • Why a fast MVP says nothing yet about the cost of verification and production operation;
  • How, according to Albina, her team built a RAG-based investment assistant in 2023, when the ready-made tooling was only beginning to emerge;
  • Why an assistant “for the entire bank” is not one large prompt, but routing, data, permissions, tools, memory, security, and different owners of quality;
  • What the experience of the Oleg assistant can offer modern LLM products.

For me, the main question of the stream is this: is AI Product Builder a new box on the org chart, or a new minimum level of ownership from the problem through proof of quality?

Bring your questions. Examples are especially interesting where a product manager already builds prototypes and evals personally, while an MLE increasingly participates in selecting the user scenario and product trade-offs.

#AI #Product #MachineLearning #Evals #Engineering #Agents