François Chollet Leads New Initiative to Test AI | Gpu vs cpu reddit | Hardware components of computer | 10 parts of computer | Turtles AI

François Chollet, a former Google engineer, co-founds the ARC Prize Foundation, a nonprofit to develop benchmarks to assess AI’s ability to achieve human-like intelligence. The organization will develop new competitions and advanced tests for AGI.

Key Points:

• Creation of the ARC Prize Foundation by François Chollet to promote progress in general AI (AGI).
• Introduction of innovative benchmarks to measure basic human capabilities in AI.
• Collaboration with industry leaders to improve AGI standards.
• Launch of new competitions and benchmarks for 2025.

François Chollet, a renowned AI researcher and former Google engineer, has announced the creation of a new nonprofit organization, the ARC Prize Foundation, with the ambitious goal of redefining the way general AI (AGI) is evaluated. The term AGI, while controversial, typically refers to AI that can tackle complex tasks that go beyond the limits of training data. Led by Greg Kamradt, a former CTO at Salesforce and now president of the organization, the foundation aims to set new standards through targeted benchmarks and global competitions.

The heart of the project revolves around the expansion of the ARC-AGI, a test Chollet created in 2019 to evaluate the ability of AI to solve problems never seen before, using only acquired skills and the ability to adapt. The approach is distinct from traditional benchmarks that reward AI that can solve complex but specific problems. Indeed, ARC-AGI proposes tasks in the form of visual puzzles, designed to measure the ability of AIs to understand and create novel patterns. Despite some progress, the most advanced AIs, such as OpenAI’s o3 model, have shown significant limitations, often compensating for a lack of intuition with the use of enormous computational resources. Chollet emphasizes that a truly general AI should excel at such tests efficiently and without relying on massive computations.

In 2025, the ARC Prize Foundation will introduce a second generation of ARC-AGI benchmarks and launch a competition aimed at inspiring significant developments in AGI. In parallel, it plans to launch an academic network to foster international collaborations and share advances in the field. The organization also intends to design a third benchmark that addresses some of the weaknesses of current tests, such as the susceptibility of AIs to brute-force approaches.

As the scientific community continues to debate the very definition of AGI, the ARC Prize Foundation aims to bridge the gap between human and machine capabilities, not just superhuman performance. Despite criticism, Chollet insists that the ultimate goal is to create tests that are easy for humans but challenging for machines. It will be a turning point when AIs achieve human-like results without the need for special training.

With a focus on transparency and collaborative progress, the foundation is already building a coalition with leading AI labs, including potential partnerships with OpenAI. OpenAI CEO Sam Altman recently expressed interest in working together to develop new AGI standards, raising expectations for the future evolution of the field. The ARC Prize Foundation represents an ambitious and bold initiative to address the challenges of AI research.

With the launch of the second generation of ARC-AGI, 2025 could mark a new era in understanding and measuring general AI.