Hello Deep Learning • Bert Hubert • GOTO 2023
Interesting. speech Bert Hubert with a story about Deep Learning on his fingers. Bert is enough. famous A friend who is a bit of a scientist, developer and entrepreneur. It’s funny how he initially says he’s been ignoring hype around deep learning for years, in part because blockchain was also fervently preached. But after chatGPT came along, it was impossible to ignore deep learning, and he decided to dive into the domain. For this, he chose an approach that is similar toKubernetes the hard way" by Kelsey Hightower. To do this, Burt took a standard number recognition problem and decided to solve it from scratch:) And then he told me briefly about it.
- Attention is all you need NIST and its handwritten numerals as a learning base The approach is to subtract the pixel values for 3 and 7 and the separation of the multitude
- Further complicated with multiplication by random matrix Then we add the loss function. (loss function)
- Moving to three layers to classify numbers into ten categories n
- Addition of nonlinearity in multiplication (ReLU and GELU) Adding gradients to train model coefficients and getting good results Adding noise to the original data and getting a full thrash as results from the trained model and then the author says the following:
Production use of a neural network tends to go through these four phases (if you are lucky):
It works on the training data
It also works on the validation data
After a lot of disappointment, we get it to work on other people’s real life data too
Other people can get it to work on their own data as well
Almost all demos declare victory after phase 2.
And then the author makes the final conclusions on the speech. Deep learning is real. Deep learning allows you to get magical results Deep learning is not magical or deceptively simple.
- You can still get on board this ship:)
As a result, the author suggests not to rely on external APIs of cloud services, but to make their products on top of solutions available on-premise.
The author offers to look at the following materials
- Whisper.cpp: state of the art voice transcription in dozens of languages, entirely self-contained on your own computer/phone: https://github.com/ggerganov/whisper.cpp
- LlaMA “GPT-like”, self-contained, own computer etc: https://github.com/ggerganov/llama.cpp
- https://berthub.eu/articles/posts/hello-deep-learning/ - the series behind this presentation, https://github.com/berthubert/hello-dl
- https://berthub.eu/articles/posts/ai-is-guaranteed-to-disrupt-us/
#ML #DataScience #Data #Math #Software #SoftwareDevelopment #Engineering