Google "We Have No Moat, And Neither Does OpenAI" (from semianalysis.com)
Interesting. document About the future of AI from one of the researchers at Google, who decided to remain anonymous (since it is claimed that this is a leak). It is interesting to read it in its entirety, but the main idea is that
- Open-source LLM models are developing rapidly, overtaking the development of closed-source models. (For example, OpenAI and Google) And getting closer in quality. The bottom line is that the community can conduct more diverse experiments and, by achieving results in one direction, the following experimenters can use these achievements. (Part of what makes LoRA so effective is that - like other forms of fine-tuning - it's stackable)
- The quality of the data can be more important than the number of parameters in the model, and if the model is smaller, the iteration length in the experiments is smaller and the fine-tuning cost is smaller. The author says about the use of "small, highly curated datasets", which are also often in open source.
- Competing directly with the open-source movement is not worth it, and if you can not prevent, then lead. The author recalls the platform and says that you can do the same with Chrome or Android, where projects in open-source and Google acts as "a thought leader and direction-setter, earning the ability to shape the narrative on ideas that are larger than themselves."
P.S. Separately interesting to read. article Comparison of the quality of models to which the author appeals.
#ML #DataScience #Software