Why ask an AI to write a response when it can decide in 20 milliseconds?

With joey-1, Baseten offers an open 2-billion-parameter AI model that makes decisions in 20 ms, starting at $0.01 per million tokens.

Should a refund request be approved? Does a message deserve to be flagged? Which team should a support ticket be routed to? For these day-to-day operations, companies often use language models capable of drafting several paragraphs, even though a simple decision would suffice. Baseten proposes to eliminate this step with joey-1, a small artificial intelligence model designed to respond directly with a choice and a confidence score, without generating text. The announced result: a median latency of 20 milliseconds and a price tag of just $0.01 per million tokens.

Developed by Base Labs, Baseten’s research lab, joey-1 belongs to an emerging category: decision models. Their goal is not to compete with large conversational models on writing or open-ended reasoning, but to quickly execute the structured decisions involved in running applications. Artificial intelligence doesn't always need to talk

A traditional language model processes a prompt and then generates a response in the form of text, typically word by word. Even when simply asked to choose between two options, it mobilizes a generation mechanism intended to produce language.

For repetitive tasks, this approach represents an unnecessary cost. A customer service application, for example, does not necessarily need a detailed explanation to determine whether a request concerns billing, a technical issue, or a cancellation.

Decision models take a different approach. They receive inputs, called state, along with one or more questions defining the possible choices. They directly return structured responses and probabilities associated with each option.

With joey-1, this operation is carried out in a single compute pass, without the progressive generation of a free-form response.

The model can handle several types of decisions: selecting a category from multiple options, answering a closed question, or assigning a score based on a predefined scale.

It also accepts images as input. Developers can therefore consider visual classification, content moderation, or content routing applications beyond just text processing.

The value lies not in the system's ability to explain its choice at length, but in its capacity to provide an output that is directly actionable by software. Twenty milliseconds for a decision

The performance highlighted by Baseten involves an average request of 2,000 tokens. On its production infrastructure, joey-1 boasts a median end-to-end latency of 20 milliseconds.

This means that half of the requests under the evaluated protocol are processed in 20 milliseconds or less. However, this metric, known as P50, does not describe maximum delays or the performance systematically achieved across all environments.

By way of comparison, elephant-1, the mid-sized model in the same family, reaches 49 milliseconds according to metrics shared by Baseten. Joey-1 thus proves to be approximately 2.5 times faster.

The company also presents it as five times faster than certain comparable decision models, though it has not yet published all the technical details required to replicate this comparison.

This speed is especially advantageous when multiple decisions must be made sequentially. In an application that uses different models to classify a request, verify its compliance, assess its urgency, and select a resolution path, every additional call extends the total delay.

Reducing these operations to a few dozen milliseconds can therefore have a direct impact on service responsiveness. One million tokens for a penny

The other selling point involves cost.

Baseten bills joey-1 at $0.01 per million input tokens. Because the model does not produce traditional generative text, the pricing schedule lists a zero cost for structured output tokens.

The company gives the example of a service processing ten million support tickets per month, with approximately 500 tokens per request.

Such a volume represents five billion tokens monthly. At this rate, the theoretical processing cost comes to $50 per month with joey-1, compared to $500 with mammoth-1, the 27-billion-parameter model offered by the same lab.

These amounts reflect the inference cost indicated in the pricing schedule. They do not necessarily include expenses related to development, integration, human supervision, or other services used within a complete application.

Nonetheless, the gap illustrates the economic value of a specialized model. For companies conducting millions of classifications every day, the unit price of a decision can become as important as its accuracy. Two billion parameters, an assumed specialization

Joey-1 is built on Qwen3.5-2B, a two-billion-parameter model to which Base Labs applied additional training focused on decision-making tasks.

This specialization transforms a general-purpose model into a system capable of directly selecting structured responses and assigning probabilities to them.

The lab used a common training method across three models of different sizes: joey-1 at 2 billion parameters, elephant-1 at 9 billion, and mammoth-1 at 27 billion.

The family is designed to address different tradeoffs between speed, cost, and the ability to handle complex decisions.

Joey-1 prioritizes high volumes and latency-sensitive operations. Elephant-1 represents a mid-range solution, while mammoth-1 targets decisions requiring more knowledge or higher accuracy.

All three models share the same programming interface, allowing users to switch from one to another without deeply modifying an application.

In particular, Baseten suggests using joey-1 as a first tier of processing, and then escalating ambiguous situations to a more powerful model.

This setup avoids systematically deploying the most expensive resources for simple requests. A performant small model, but not the smartest in the family

In the evaluations presented on its Hugging Face model card, joey-1 achieves a score of 63.30 on the public benchmark of the Jev Decision Index 0.4.

This result places it 37th out of 230 evaluated models and,