[2/2] Hands-On Large Language Models (AI column)
Continue. story As for this book, it consists of three parts and 12 heads (I've read four and a half chapters so far, so I'll talk about them in more detail.). But in short, the book begins with the basics of LLM, then the authors move on to their use for solving problems, and end with the methods of teaching models.
Part I: Understanding language models The first part lays the foundation, explaining what language models are and how they work. Head. 1. Here is an overview of the development of language processing - from "bags of words and classical representations of text to the emergence of methods such as word embeddings and transformer architecture. The reader will learn what large language models are and how they differ from previous approaches. The chapter also discusses why LLMs have become so useful. Head. 2. The authors are immersed in the topic of tokenization and embedding - key concepts for representing the text in numerical form. The authors describe in detail how the LLM tokenizer works (tokenize)Compare different types of tokens (words, words, symbols, bytes) and demonstrate the properties of trained tokenizers. Next, we consider how the model turns tokens into vector representations (embedding) and how these emblems can be used to represent words, sentences and documents. Examples of how to measure the semantic proximity of texts with the help of embeddings are given. The same chapter explains traditional algorithms for constructing embeddings. (For example, word2vec) and more modern approaches, up to how LLMs themselves form emblems while working. The chapter concludes with a practical example: teaching an embedding model for a recommendation system (For example, the recommendation of songs on the similarity of embeddings). Head. 3. The authors show the insides of the transformer, explaining what is happening under the hood. They talk about forward-pass models: how input tokens are processed, how the attention matrix is calculated. (attention) and how the model chooses the next token. An important part of the chapter is an intuitive explanation of the self-attention mechanism. (focus)The model takes into account the context of words in generation. Optimizations are also discussed: how models handle multiple tokens in parallel, what maximum context size is supported, and how key and value caching speeds up text generation.
Part II: Use of pre-trained models The second part is devoted to practical ways of using ready-made language models and embeddings in various text processing tasks. Here, the authors move from the design of models to their use “out of the box” for solving applied problems. Head. 4. Classification of text Head. 5. Clustering and thematic modelling (BERTopic) Head. 6. Industrial engineering (Chain of Thought, ReAct, Tree of Thought) Head. 7. Advanced Text Generation Techniques and Tools (LangChain and Agents, Memory, Tools) Head. 8. Semantic Search and Retrieval-Augmented Generation (RAG) Head. 9. Multimodal models (text + pictures, CLIP, BLIP-2)
Part III: "Training and fine-tuning models" The last part of the book is devoted to how to create your own models and adapt existing LLMs to specific tasks. Here the material becomes more advanced. n Head. 10. Creating embedding models for text Head. 11. Fine-tuning representation models for classification Head. 12. Fine-tuning generative models
In the final part (Afterword) The authors sum up the progress made and look into the future development of large language models. They note the enormous and wide-ranging impact that LLMs have already had on the world and that understanding how they work opens up new opportunities for professionals.
#Architecture #Software #AI #Engineering #ML #Data #SystemDesign #DistributedSystems