OpenAI has released the first performance results from Jalapeño, its custom-designed inference chip, showing significant gains in both speed and energy efficiency compared with existing AI hardware.
According to OpenAI, Jalapeño is designed to handle modern AI workloads while reducing the traditional trade-off between high throughput and low latency. In practical terms, this could mean faster responses for users, more responsive AI agents, and greater capacity to serve growing demand without a proportional increase in energy consumption.
A New Approach to AI Inference
OpenAI evaluated Jalapeño using InferenceX, a public benchmark developed by SemiAnalysis that measures the performance of complete AI-serving systems rather than focusing solely on individual chips.
The company tested the architecture with three large open-weight models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Across these workloads, Jalapeño reportedly achieved between 1.5 and 1.9 times more AI work per watt at peak throughput, while end-to-end latency was reduced by between 1.7 and 3.6 times.
The results are particularly relevant for interactive AI applications, where even small delays can accumulate as an agent performs multiple steps. In these workloads, OpenAI says Jalapeño delivered between 2.1 and 4.1 times higher performance.
Designing the Entire System Around AI
Jalapeño was not developed simply as a faster processor. OpenAI designed the chip together with its memory, networking, software, and rack-scale infrastructure.
This approach addresses one of the central challenges of AI inference: different stages of model execution place different demands on hardware. During prefill, processing the user's prompt is primarily compute-intensive. During decoding, when the model generates a response token by token, memory bandwidth becomes increasingly important.
Communication between chips can also introduce delays. Jalapeño therefore emphasizes keeping model data close to the computing resources that need it, reducing unnecessary movement and communication.
The architecture is intended to provide a more balanced environment for both prefill and decoding, while remaining flexible enough to support different model architectures and increasingly agentic workloads.
AI Helped Build the Chip
One of the most notable aspects of the project is that AI itself played an important role in Jalapeño's development.
OpenAI says its teams used AI to explore hardware implementations, accelerate design and verification cycles, and optimize parts of the chip's arithmetic circuitry. The company moved from the initial design to tapeout in approximately nine months.
The chip was also designed to be relatively predictable from a programming perspective, making it easier for both human engineers and AI systems to optimize workloads across the architecture.
OpenAI reports that, using Codex and GPT-Astra, engineers were able to optimize three open-weight models that were not originally part of the chip's production plan within two months. For selected attention and mixture-of-experts components in GPT-OSS, AI-generated implementations were reportedly 1.5 to 1.8 times faster than existing implementations written by human experts.
These figures apply to specific components rather than complete models, but they illustrate a potentially important development cycle in which AI can help design hardware and subsequently optimize the software running on it.
Strong Results Across Different Models
Jalapeño showed competitive results across all three models tested.
For GPT-OSS 120B, OpenAI reports approximately 1.9 times higher peak throughput per watt than the comparison system, alongside roughly 1.7 times lower end-to-end latency.
On DeepSeek R1 670B, the chip achieved approximately 1.7 times higher peak performance per watt and 3.6 times lower end-to-end latency.
The results for Kimi K2.5 1T, the largest model included in the public testing, showed around 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency.
Together, these results place Jalapeño on what OpenAI describes as the Pareto frontier across the tested operating range, meaning it provides a particularly strong combination of performance and energy efficiency rather than optimizing only for one metric.
What Jalapeño Could Mean for AI Infrastructure
The importance of Jalapeño extends beyond benchmark numbers. As AI models become larger and AI agents perform increasingly complex tasks, infrastructure efficiency is becoming a central part of the economics of AI.
More useful computation from the same amount of power and hardware could allow AI providers to serve more users while controlling infrastructure costs. Lower latency could also enable new applications in which rapid, repeated model interactions are essential.
OpenAI says Jalapeño could make ultra-fast inference more efficient, improve the economics of fast inference, and increase the efficiency of batch workloads.
The Beginning of a Longer Hardware Roadmap
OpenAI plans to begin deploying Jalapeño within its own computing infrastructure by the end of 2026. The company describes the chip as the first generation of a broader, multigenerational platform, with a second generation already in development and a third generation taking shape.
At the same time, OpenAI says it will continue using accelerators from NVIDIA and other partners for both training and inference.
Jalapeño therefore represents not a replacement for the broader AI hardware ecosystem, but an additional layer of OpenAI's strategy to control and optimize more of the infrastructure behind its AI products.
The larger goal is clear: as demand for AI continues to grow, improving inference speed and energy efficiency will be essential. Jalapeño is OpenAI's attempt to address that challenge by designing hardware, software, networking, and AI models as parts of a single system.