[1/2] François Chollet: How We Get To AGI (AI column)

I watched with great interest. speech Francois Schollet at the Y Combinator AI Startup School conference in San Francisco. Francois is the creator of the popular deep learning library. Keras (2015 year)It has become one of the key tools. In 2015 He joined Google for nearly a decade, researching computer vision and artificial general intelligence. In 2019 The year he created the benchmark. ARC-AGI To measure the capacity of AI for general intelligence. In 2024 Scholle launched ARC Prize Competition with a prize fund of $1 One million dollars for ARC-AGI. In November 2024 Scholle left Google to start a new startup. In January 2025 He founded a research laboratory. Ndea Together with Zapier co-founder Mike Noop, she focuses on developing AGI through software synthesis.

In this speech, Francois shared some really interesting ideas. 1. Evolution of AI paradigms Scholle traced the history of AI, starting with the constant decline in the cost of computing by two orders of magnitude every decade. 1940 years. In 2010The paradigm of scaling the pre-training of large language models, based on increasing the size of models and the volume of data, became dominant. However, in 2024 There was a major shift in adaptation during testing. (Test-Time Adaptation, TTA). This technology allows models to dynamically change their behavior based on the specific data they encounter during output. TTA includes methods such as learning during testing, software synthesis, and the synthesis of thought chains. 2. Definition of intelligence François criticizes the traditional understanding of intelligence as the ability to perform specific tasks. Instead, he proposes a definition of intelligence as the efficiency of converting past experiences into skills to solve future problems with high novelty and uncertainty. The key difference is static skills versus mobile intelligence. Static skills are learned programs for solving known problems, whereas mobile intelligence is the ability to synthesize new programs for unfamiliar tasks. 3. The kaleidoscope hypothesis One of the central concepts of the speech is the kaleidoscope hypothesis. Scholle argues that the world's seemingly infinite complexity is actually made up of a small number of "atoms of meaning" - fundamental building blocks that recombine in different situations. Intelligence is the ability to extract these reusable abstractions from experience and effectively recombine them to solve new problems. This explains why simply scaling pre-training didn't lead to AGI - models lack the ability to recombine on the fly. 4. Two types of abstraction Francois identifies two fundamental types of abstraction. - Type. 1 (value-centric): works with continuous spaces, uses distance functions to compare objects. It is the basis of perception, intuition and modern machine learning. **- Type. 2 (software-centric)**Works with discrete programs and graphs, looking for exact structural matches. It is at the heart of human thinking and planning. Transformers do a great job with abstractions of the first type, but do not work well with abstractions of the second type. To achieve true intelligence, both approaches must be combined. 5. Software synthesis and discrete search The key to overcoming the limitations of modern AI Scholle sees in software synthesis - a method that allows you to automatically create programs to solve problems. Unlike gradient descent, which requires huge amounts of data, software synthesis is extremely data efficient but faces a combinatorial explosion. The solution is to use deep learning intuition (type 1) for discrete search of programs (type 2)By creating a system that can effectively explore program space.

Continuation about frameworks and laboratory Ndea into post.

#AI #ML #Software #Architecture #Processes