Richard Sutton founds Oak Lab for agents that learn continuously

Richard Sutton fonde Oak Lab avec Khurram Javed pour créer des agents d'IA apprenant en continu avec 20 watts, rejetant l'apprentissage profond classique.

Richard Sutton, co-founder of modern reinforcement learning and winner of the 2024 Turing Award with Andrew Barto, has left Keen Technologies, John Carmack's AGI company, to found Oak Lab with his former PhD student Khurram Javed. The new Canadian entity ranks among those "neolabs" that are betting on an algorithmic breakthrough rather than scaling up.

Sutton praises Keen but outlines his divergence: he considers current deep learning methods to be weak and inefficient, and believes they require not mere tweaks but a fundamental overhaul before they can serve as the foundation for AI's highest ambitions. Oak Lab's bet is to build agents that learn from their own experience, continuously, rather than imitating datasets prepared by humans.

The technical argument is developed in a post published the same day. Common optimization algorithms, such as stochastic gradient descent, assume that all data is useful: they assign credit for an error to all parameters involved, without distinguishing between what is predictable and what is not. As a result, when faced with a real, noisy stream, they absorb the noise instead of learning the signal. The researchers demonstrate this on a set of handwritten digits drowned in parasitic pixels: where the classic method reinforces connections across the entire image, theirs only reinforces those in the information-bearing area.

From this stems the lab's direction: algorithms that learn to assign credit, capable of working on a noisy stream, in real time, without storing or replaying data, using what they call single-batch learning. The stated Holy Grail is quantified: a one-trillion-parameter agent that learns and plans in real time on twenty watts, which is the order of magnitude of a human brain.