Machine Learning System Design (Category ML)

While I was resting on the Putorana Plateau without the Internet, I read a book.ML System Design" from Valery Babushkin and Arseny Kravchenko. It was interesting to me for two reasons: The theme of classic system design And I'm interested in machine learning.) I like the idea of the authors that the design of systems (ML systems including) This is a multi-stage process, where skills from various areas are useful, which should be used within the framework of a systematic approach. For ML, this multi-step process involves much more than just choosing the right model/algorithm.

If you compare conventional and ML design, there are moments that they differ from each other. 1. Determinism vs probability We try to design conventional systems with a certain degree of determinism, and we cover them with tests to test predictability. ML systems are by design probabilistic. In conventional systems, we usually have a clear algorithm that turns inputs into weekends, and the entire non-deterministic part is usually on the operating side. (work under load, reliability and failure). In ML systems, the execution model itself is indeterministic, which adds to the complexity of assessing the quality of its work. 2. One and Two-Phase Architecture ML systems usually have two very different phases of operation: training and inference. In conventional applications, we do not have a training phase - it is already trained by engineers to do the necessary work. (Aka working inference mode). 3. Data dependency In traditional development, we usually have structured data that applications work with – in design, we design data models and model their relationships, often OLTP systems. In ML systems, we have much less control over data - we have to city pipelines and often work with OLAP constructions while learning to work in a format closer to traditional systems in order to have time to make Inference with an acceptable response time.

The authors devote much time and space to discussing architectural considerations. - Data conveyors Building a robust data pipeline is becoming a fundamental component of the architecture, supporting data absorption, real-time processing, and scalable storage solutions. - Modular design The need to separate ML components from the basic logic of the application, which allows you to independently update the models without disrupting the main system. - Monitoring and monitoring Traditional logging systems must evolve to include specific ML metrics such as data drift and model performance.

Separately, I note an interesting format of stories at the campfire. (campfire stories)Where authors share their stories from the past, talking about solving real problems from their experience. It looks interesting and similar to when we tell stories by the fire over a mug of hot drink. In general, I found the book interesting and useful - it is useful to read both ML engineers and classical software engineers, who increasingly have to design Gen AI applications.

P.S. The authors have their own tg channels, where they write about ml and not only about ml.

• Valera - @cryptovalerii
• Arseny - @partially\ unsupervised

#ML #AI #Engineering