Emerging Patterns in Building GenAI Products (AI column)

Interesting. article Martin Fowler and Bharani Subramaniam, in which the guys talk in pattern format about the basic cubes from which GenAI applications are built. This article is typical of the current Fowler - it is nothing new at all, but it describes quite clearly what is already known at the moment:) Currently, the list of patterns is only those listed below, but the authors promise to gradually replenish it.

- Direct prompting Prompting from the foundation LLM application. I’m not sure that this is a pattern, but the authors of the campaign are sure of it. I think it's kind of like calling a function a pattern:) - Embeddings - here the authors talk about the translation of the text or picture in the form of a multidimensional vector so that close in essence were close in this multidimensional space. The metric of proximity can be different, but it is often cosine similarity when we look at how well vectors are aligned. In the example, the very transformation of text to embedding occurs through the call of the library function. (And engineers don't have to think about how it works under the hood.). The bottom line is that these embeddings are also needed by the LLM itself and can be further used to search the vector database. A Fun & Absurd Introduction to Vector Databases - Alexander Chatzizacharias - GOTO 2024The one I mean. told earlier - Evals Here the authors talk about the evaluation of LLM responses in the context of specific tasks. In fact, this is like testing conventional software, but with an asterisk. The asterisk is due to the fact that the LLM response is not deterministic, which means that it will not work to test the “request-answer” in the forehead, and we need to do something like load testing, when we test the performance model in terms of semantics and accuracy of the response. This requires scenarios + expectations from the answer + what backlash we allow in deviations from this answer. - Hybrid retriever - here the authors say that it is possible to combine answers from a vector database when searching for close entities in terms of embeddings with a more canonical search, conditionally, full text - Query rewriting - You can use the LLM#2 in order to rewrite the pump and then several of its options to feed the LLM#1. This in theory allows you to improve the prompt, and LLM#1 LLM#2 They can be the same LLM or different. For example, the LLM#2 can be trained to make good prompts out of bad ones, and the LLM#1 high-quality - Reranker When searching for data, many documents can be returned, and the context of the models is not yet rubber, so there is an option to separately rank the found documents and only the necessary ones to throw into the context of the query for the LLM. - Retrieval augmented Generation (RAG) This pattern is that you can not just ask LLM to generate an answer, but also allow it to search for information, and then throw the information found into the context of the query. Within this framework, we can use Hybrid retriever, Reranker, Query rewriting

About faintuning models promised to write in the next series, perhaps, then write something about the agents.

In general, the article is useful for engineers to understand the basic primitives of GenAI applications.

P.S. I learned about the article from the @startup\ architecture channel of my colleague, Anton Skogorev, who is a techdir at our AI Center.

#AI #ML #Engineering #Software #Architecture #SystemDesign #DistributedSystems