Skip to content
#ML

Operationalize a Scalable AI With LLMOps Principles and Best Practices (Category ML)

#ML #AI #Software #Architecture #Future

I have long noticed that the fashionability of the theme can be tracked by the presence of a bundle of somethingOps, here came the turn and LLMOps, about what This is according to a recent article from DZone.. I used to be on the hype.

  • DevOps - It's been a long time. (More in the book "Accelerate", about which I told)We are talking more about Platform Engineering.
  • DevSecOps - a hot topic, more in the book "Agile Application Security", about which I told recent “Secure by Design at Google” handler separately
  • DataOps - a long-standing and relevant topic about the built processes of working with data. They are needed as prequisites for effective work on ML models.
  • MLOps This is a very relevant topic that includes a set of practices that combine ML, DevOps and data engineering, which are aimed at creating, deploying and operating ML systems in the market reliably and efficiently. I wonder if we're on this subject. stripper Podcast "MLOps in Theory and Practice" More information about XOps approaches can be found in the report "The Pipeline-Driven Organization", about which I have already told

In the same article, we are talking about a subset of MLOps approaches that focus specifically on LLM applications, which are now a hot topic. The following article first defines LLMOps. Further, we understand the difference between MLOps and LLMOps by the criteria on which the main focus is, how model adaptation looks, model quality assessment, model management, including versioning and metadata, how models are deposited and how their work is monitored. Next, the author analyzes the basic characteristics of the LLM

  • That they exist in different forms: proprietary models with paid APIs, pre-training models, fine-tuned models That there is so-called prompt engineering, as many LLMs accept natural language text as input Sometimes you can add context to user requests. (context-based prompt engineering:)To make them more efficient, using new tools like vector databases. (recently note) They are quite large, sometimes hundreds of gigabytes, and they may also require a GPU not only for training, but also for processing real-time requests. Evaluating their quality is quite difficult, so it is often necessary to build human feedback directly into the MLOps process for evaluating and testing models.

The key points of LLM applications right now are the following: Prompt engineering, which we talked about above

  • RAG (retrieval augmented generation), which usually relies on the already mentioned vector databases for semantic search, as well as on the feature store where features are stored. Fine-tuning LLMs are the process of adapting a pre-trained LLM to a relatively small dataset specific to a particular field or task. Pre-training a model from scratch is something about the rich process of learning a language model on a large dataset. (e.g. text, code) without using any prior knowledge or weights from the existing model:)

Next, the author shows the reference architectures of LLM applications with RAG from Databrics, and finalizes the pros and cons of LLMOps

Minimal changes to base model – most LLM applications can be run on top of base models with minor changes Easy to model and deploy - LLMOps makes it easier to use models Advanced language models – you can start using complex models through the API and then go to open source models Human feedback – in the case of LLM, feedback from people is necessary, which adds to the complexity Limitations and quotas – when using external APIs, you need to understand their limitations and cost of use Risky and complex integration - when using external APIs, you need to understand what data you share and how much it is OK

#AI #ML #Software #Architecture #Future