More responses per watt: OpenAI’s first processor posts its initial results

Jalapeño, OpenAI’s first inference processor, promises higher throughput and lower latency ahead of deployment by the end of 2026.

Producing more responses while reducing wait times and power consumption: OpenAI has published the first results for Jalapeño, its first processor designed specifically for large language model inference. Deployment within the company’s infrastructure is scheduled to begin before the end of 2026.

Inference is the stage at which an already trained model processes a request and generates a response. Jalapeño is therefore not presented as a processor for training OpenAI’s future models, but as a chip for running them once training is complete.

This specialization directly targets ChatGPT, Codex, agents, and API services. OpenAI says users could see faster responses, more responsive Codex sessions, and more reliable access as demand increases. The company has not provided a precise rollout schedule for individual products, models, or regions.

The results were obtained using three open models with different sizes and architectures: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. This selection allows OpenAI to demonstrate that Jalapeño is not limited to its own internal models.

Across the three workloads, the chip reportedly delivered between 1.5 and 1.9 times more throughput per watt at each system’s most efficient operating point. End-to-end request latency was reduced by between 1.7 and 3.6 times.

The gains were larger under highly interactive workloads. Jalapeño reportedly generated 1,459 output tokens per second per user with GPT-OSS 120B, compared with 535 for the competing system. The respective results were 700 versus 169 with DeepSeek R1, then 694 versus 182 with Kimi K2.5.

These measurements represent conditions optimized for each platform. They do not mean that every ChatGPT response will be generated four times faster. The delay experienced by an individual also depends on the selected model, its reasoning time, current traffic, routing, tool calls, and the user’s connection.

The comparisons use InferenceX, a public benchmarking platform developed by SemiAnalysis. It evaluates the complete request-processing cycle, from reading a prompt to generating the answer, across different levels of throughput and responsiveness.

For GPT-OSS 120B, OpenAI compares Jalapeño with an NVIDIA GB200 system. DeepSeek R1 and Kimi K2.5 are compared with a GB300. The tests use a nominal workload of 8,000 input tokens and 1,000 output tokens in a configuration designed to process multiple users simultaneously.

At each system’s most efficient operating point, Jalapeño reaches 85,448 mixed tokens per second per kilowatt with GPT-OSS, compared with 44,960 for the GB200. The difference is 19,641 versus 11,781 with DeepSeek R1 and 18,195 versus 11,862 with Kimi K2.5.

The much larger differences highlighted by OpenAI, sometimes exceeding factors of 50 or 100, correspond to a specific scenario. Jalapeño is constrained to the competing system’s maximum decoding speed, after which both platforms are compared at that same level of responsiveness. Those figures do not represent a general performance lead across every workload.

Power consumption also requires careful interpretation. Jalapeño has a stated thermal design power of 700 watts, while the competing systems are counted at 1,200 or 1,400 watts depending on the model being tested. OpenAI says its chip remained below 550 watts under sustained load during the tests.

Performance was nevertheless normalized using the published maximum power rating for each accelerator rather than identical measurements of actual system-wide electricity consumption. The results do not directly account for all the energy used by host processors, external memory, networking, cooling, or the data center itself.

They also do not provide a total cost-of-ownership comparison. Jalapeño is not sold separately, and OpenAI has not disclosed its purchase price, manufacturing cost, production yield, or large-scale operating expenses. Better throughput per kilowatt is not enough to determine the actual cost of each response.

The measurements were produced and presented by OpenAI. InferenceX provides a public and reproducible framework for platforms available through its standard procedures, but Jalapeño remains internal hardware. No independent laboratory has yet announced that it reproduced the results on external infrastructure.

The chip’s technical value comes largely from reducing data movement. Running a language model involves several phases with different constraints. Initial prompt processing depends mainly on compute capacity, while progressive response generation relies more heavily on memory access speed.

When a model is distributed across several chips, moving data and coordinating between units can create periods of inactivity. A processor that performs one operation quickly may lose part of its advantage if information must constantly travel among memory, cores, and the network.

Jalapeño was designed to retain locally some of the state required for generation, including the KV cache that stores previously processed context. The architecture can then activate different combinations of compute, memory, and communication resources depending on the current request phase.

Networking is integrated into the design. The aim is to keep an entire workload within a single connected system and reduce transfers between distant resources. This structure is intended to handle both long-context processing and rapid response generation.

The platform was developed with Broadcom and Celestica. OpenAI defined the architecture and requirements, while Broadcom contributed to chip implementation, networking, and industrialization. Celestica is involved in board and rack-system integration.

The move from initial design to tape-out, the stage preceding physical manufacturing, reportedly took nine months. OpenAI attributes the timeline to hardware and software being developed together, as well as the use of its own models during the design process.

Those models were used to explore different implementations, shorten