Skip to content
#AI

The Engineering Unlocks Behind DeepSeek YC Decoded (AI column)

#AI #Engineering #Software #ML #Architecture

Interesting. 13 minute analysis of DeepSeek R1 From the guys at Y Combinator, who focus not on hype, but on engineering stuff. The main points of discussion are such

  1. Deepseek has announced the R1 logic model, which delivers comparable performance to OpenAIo1 at lower cost.
  2. This caused panic on social media and a decline in Nvidia’s market capitalization. 600 billions of dollars.
  3. But DeepSeek is not a new player in the market. They publish their research results and weight models, unlike other major labs like OpenAI and Google DeepMind. And many of the results have already been published before, for example, they optimized learning in fp8 and corrected the accumulation of error.
  4. It is important to distinguish between DeepSeek-R1 and DeepSeek-V3 models. DeepSeek V3 delivers performance comparable to GPT4 and other basic models. R1 is a reasoning model based on V3 and achieves performance comparable to OpenAI o1 and Google Gemini Flash 20.
  5. In V3, they used an architecture that only activates. 37 There are billions of parameters for each prediction, which saves a lot of computation and uses the technology of multi-head latent attention. (mla) reduce memory and increase bandwidth. 6 ) For R1, they came up with an interesting reinforcement learning scheme. (reinforcement learning)
  6. Part of the hype around the R1 has been linked to the model's availability through the Deepseek website and app. The model itself offered comparable performance at a fraction of the cost of other models. 8 ) Most of the hype is associated with misconceptions about the cost of training, the amount was specified for the final training model without the training.
  7. DeepSeek techniques can be replicated to create their own models, for example, the University of California, Berkeley Laboratory has applied these methods to create small reasoning models in just a few minutes. 30 dollars.
  8. Since this is a video from Y Combinator, they end up with the idea that there is room at the forefront of AI for new players who can move old-timers by optimizing GPU workloads, improving software, and so on. This reduces the cost of implementing AI in the end products, making this the right time to start a startup.

#AI #Engineering #Software #ML #Architecture