Programming meanings (AI column)

I saw something interesting yesterday. speech Alexey Gusakov, CTO of the Search and Advertising Technologies business group in Yandex. Alexey spoke about changes in approaches to creating products – they are actively developing the ML and LLM stack and introducing neural networks into key services. (Search, Alice, Browser, etc.). In short, the focus of development shifts from detailed coding of algorithms to designing intent and meaningYou formulate the task, constraints, knowledge sources, and quality metrics, and program the behavior of the system with a combination of prompts, data, tools, and rewards. (reward). In this model, value is not created by a single microservice, but by a single microservice. cycle: hypothesis → prototype → measurement → additional training / fine tuning → integration. The internal stack and processes should support this.

In fact, Alexei’s report perfectly describes the transition to programming meanings by steps. 1) How did it start? In 2022 Yandex launched a dialogue experiment “Guru on goods” – it was an attempt to turn the search into an assistant of choice: ask questions ("Which TV do you want?")The system specifies the parameters and leads to the purchase. But the users did not go - communication with the guru felt like filling out a boring questionnaire. The team spent a lot of resources, made mistakes, but received important signals about which dialogue is “selling” and which is annoying. 2) The turning point At the end 2022 ChatGPT was released and generative responses appeared in Bing. The Yandex team faced a dilemma: sawing the “big complex thing” (Ala like Bing.) Or go incrementally. We chose the second - quick, mundane improvements around the current issue. 3) What did you do? (where) We started collecting answers based on snippets and info context; we trained our own LLM for answer paragraphs. We moved on to structured answers from several sources: the model plans what documents to use and how to stitch facts. Balanced stability vs. risk: not to release a “magical” response at any cost, and move the steps, checking the quality. The focus has shifted from the “perfect recommendation” to a system of constraints: don’t repeat, preserve diversity, support newcomers, and so on. Target optimization took place within a set of rules, not on top of a black box, making it easier to control behavior. We went to assistants, but abstract descriptions like "be smart and useful" don't collect the product. We have developed principles that really work: answers are not too long/short, truthful, personalized, without fictitious products/properties; the model chooses the form of response when looking at online signals. 4) Machine learning as a conveyor Running a repeatable cycle: AI trainers mark and evaluate responses → generative model and reward model trained (RM) We are collecting feedback again. Became at meetings of the “wise” to discuss the work of models Used piplins with several models for one request: even with “frozen” parameters can grow significantly due to proper orchestration and greater calculation. 5) What problems were and how they were treated Reward-hacking: after the first RM cycle, the model “learns” to please the appraiser – suddenly lengthens the answers, begins to copy pieces of sources, inserts unnecessary disclaimers. Fixes: In the model rewards added regularization for length, fines for copypaste / office; trained stylistics. There was a funny example about disclaimers that were left only where they really help.

• Finally, Product and ML development mixed Prompts, RM, rules and metrics have become product artifacts like code. 6) Principles of ranking and answers References in SERPs are a mixture of offline relevance assessment and online success probability. Assistants build answers on top of those links, not out of thin air, to keep them verifiable.

#Architecture #Software #AI #Engineering #ML #Data #SystemDesign #DistributedSystems