Baseten creates a lab to extend the life of open models

Baseten has launched Base Labs, a research organization focused on continual learning, post-training, reinforcement learning data, and reducing the cost of running open-source models.

An open model would no longer remain frozen on the day it is released. It would continue learning, acquire new capabilities, correct certain behaviors, and become more efficient over time. That is the goal of Base Labs, Baseten’s new research organization focused on the full lifecycle of AI models.

Announced by Base Labs as a research initiative rather than a commercial product, the lab intends to publish its experiments, results, and the methods needed to reproduce them. Its work will cover continual learning, agent memory, the science of reinforcement learning, post-training, safety, and inference performance.

Base Labs is part of Baseten, a company that provides inference infrastructure for deploying and running open or custom models. That relationship gives the lab access to computing resources and real-world workloads, but it also makes the line between independent research and industrial development less clear.

The lab says it is not trying to sell a product. Its results may still support Baseten’s services. The research agenda explicitly identifies the company’s infrastructure, customer workloads, and production requirements as sources for experiments.

The boundary is therefore not absolute. Base Labs operates as an open research division inside a commercial company, with work that could benefit both the wider community and the platform funding it.

The Base Labs manifesto starts from a familiar problem: much of the knowledge required to train the most advanced models remains concentrated inside a handful of closed labs. Accessible weights do not provide the data decisions, reinforcement environments, failures, and adjustments that determine a model’s final capabilities.

Base Labs promises to publish all its results, including unsuccessful experiments. The team also commits to writing in accessible language, encouraging criticism, and working on questions whose conclusions can be tested and disproved.

A fourth principle targets methods presented as universal solutions. The lab says it will not promote any silver bullet before rigorous evaluation and will frame every finding within the limits of the experiment that produced it.

That promise goes further than releasing weights from time to time. It requires publishing configurations, data, code, intermediate measurements, and experiments that did not produce the expected result. The website does not yet specify which licenses will apply to each category of resource.

Base Labs is not beginning without previous work. Its research archive lists six publications dated between March and July 2026, before the lab’s official announcement. They examine agent memory, KV-cache compression, supervised fine-tuning, and the effects of sequential updates on a model’s knowledge.

One of its central questions concerns continual learning. Current models usually complete their training before deployment. They can receive new information inside a conversation or through a connected document database, but their weights do not change after each experience.

A continual-learning system would try to write new knowledge directly into the model without restarting the entire training process. It would also need to prevent each new update from erasing or making older knowledge inaccessible.

This problem is commonly known as catastrophic forgetting. An update intended to teach a new fact may alter the paths used to retrieve earlier knowledge. The information may not disappear entirely from the weights, but the questions that once reached it no longer lead to the correct answer.

In a study titled Can a Language Model Learn Facts Continually in Its Weights?, Base Labs writes invented facts into Qwen3 models before applying between 20 and 100 additional updates.

Training on a single statement primarily produces recitation. When the same fact is taught through several formulations, the model becomes better at applying it to different questions. After 20 sequential writes, facts learned from a single sentence retain only about 1% accuracy, compared with 46% for facts presented through more varied material.

The researchers also find that some information appears to be forgotten without being completely erased. When a fact that has become inaccessible from the weights is placed back into the prompt, the model recovers between 77% and 80% accuracy on related questions.

The study concludes that context remains a more reliable channel than the weights when knowledge must survive several updates. Base Labs is therefore not presenting a working solution to continual learning. Its own results instead show how resistant the problem remains.

Agent memory is a second area of study. During a long session, a model keeps previous information in a KV cache. This working memory grows with the conversation, eventually affecting cost, speed, and the amount of work the agent can continue performing.

The lab wants to compress this memory without reducing it to a conventional text summary. The system must preserve useful information, discard details that have become less relevant, and provide the model with a state it can reuse during later steps.

In Still: Amortized KV Cache Compaction in a Single Forward Pass, the team introduces a small module added to each model layer. It condenses the cache’s keys and values in a single pass, without running a separate optimization for every new context.

The experiments use Qwen and Gemma models with contexts ranging from 8,000 to 128,000 tokens. Base Labs tests compression ratios between 8 and 200. On RULER, the system reportedly exceeds the strongest methods in its comparison by between 8 and 22 points, depending on the configuration.

Compression by a factor of 200 does not mean the entire conversation remains intact. The module learns to create a condensed representation considered sufficient for the selected evaluations. Rare, ambiguous, or incorrectly deprioritized information may still disappear.

The results were published by the team developing the method. They