Skip to content
back to the archive page
#AI

[2/2] François Chollet: How We Get to AGI (Category AI)

#AI #ML #Software #Architecture #Processes

To finish my account of François Chollet’s talk, here are his ARC benchmarks and the Ndea lab.

6. Proposed benchmarks

  • ARC-1. Released in 2019, it contains 1000 unique tasks requiring fluid intelligence rather than memorised knowledge. The tasks rely on basic concepts such as objects, elementary physics and geometry that a four-year-old understands. Chollet notes that over five years, despite a 50,000-fold increase in pretrained model scale, ARC-1 performance rose only from 0% to 10%. He presents this as strong evidence that simply scaling pre-training does not produce fluid intelligence.
  • ARC-2. Released in March 2025, it focuses on compositional reasoning. Unlike many ARC-1 tasks that can be solved intuitively, these require deliberate thought while remaining solvable by people. Testing 400 people indicated that groups of 10 could collectively achieve 100% accuracy. Base models scored 0%, static reasoning systems 1–2%, and even advanced systems with test-time adaptation remained well below human performance.
  • ARC-3. Planned at the time of the talk for early 2026, it departs from the input–output format to assess agency: exploration, interactive learning and autonomous goal-setting. AI enters an unfamiliar environment without knowing what the controls do, what the goal is or what rules apply. Action efficiency is central: models should solve tasks as efficiently as humans.

7. The next direction: Ndea Chollet concluded by introducing Ndea, his new research lab focused on AI capable of independent invention and discovery. Its approach is a programmer-like meta-learner that synthesises a programme for each new task. That programme combines deep-learning modules for type 1 tasks, perception, with algorithmic modules for type 2 tasks, reasoning. A global library of reusable abstractions evolves as the system learns new tasks. The first milestone is to solve ARC-AGI with a system that initially knows nothing about the benchmark.

Chollet’s vision is impressive. I really enjoyed the talk, learned a lot and added plenty of material to my reading list.

#AI #ML #Software #Architecture #Processes