ChatGPT from Scratch: How to Train an Enterprise AI Assistant • Phil Winder • GOTO 2023 (Category AI)

An interesting talk by Phil Winder, CEO of Winder.AI and author of “Reinforcement Learning”. He introduces ChatGPT and shows an entertaining demo: fine-tuning a model to write lyrics, then reading them over music generated by Stable Audio. As for the main content:

• He begins with a short history of LLMs, tracing their evolution from early forms to advanced architectures such as GPT-4 and their influence on natural language processing (NLP).
• Next come the basic architecture and components of large language models: encoder-decoder transformers, followed by decoder-only architectures such as GPT.
• He discusses preprocessing domain-specific datasets to build specialized, effective language models.
• He then covers efficient and economical training strategies, from fine-tuning pretrained models to training from scratch.
• Deployment recommendations include cloud services such as AWS and Azure, accompanied by a demo.
• Finally, he considers security and ethics when deploying LLMs in a business setting, including data privacy and model interpretability.

#AI #ML #DataScience