OpenAI and Broadcom published the first benchmark results for OpenAI's Jalapeño chip on August 25, 2026. On SemiAnalysis' InferenceX suite, the 700-watt inference accelerator delivered 1.5 to 1.9 times more work per watt and 1.7 to 3.6 times lower end-to-end latency than the tested NVIDIA GB200 and GB300 systems.
These are early laboratory measurements, not production-fleet results. SemiAnalysis observed the runs in OpenAI's lab, but says OpenAI supplied all figures and that it did not execute the full InferenceX suite. The test also used a nominal 8,000-token input and 1,000-token output rather than SemiAnalysis' longer, multi-turn AgentX workload.
Jalapeño is still engineering silicon. OpenAI plans limited deployment by the end of 2026 and a broader ramp during 2027, when NVIDIA's newer Vera Rubin platform will be a more current comparison. The published numbers therefore establish a credible first-generation contender, not a permanent performance lead.
What is OpenAI's Jalapeño chip?
Jalapeño is OpenAI's first custom Intelligence Processor, a clean-sheet accelerator for serving large language models rather than training them in production. OpenAI designed the architecture around model kernels, memory movement, networking, scheduling, and the operating patterns behind ChatGPT, Codex, the API, and agentic products. Broadcom implemented the silicon and contributes Tomahawk networking, while Celestica supports board, rack, and system integration. The companies announced a 10-gigawatt accelerator collaboration in October 2025, unveiled Jalapeño on June 24, 2026, and published the first detailed measurements on August 25. OpenAI says the design moved from initial architecture to manufacturing tapeout in nine months, with AI models assisting implementation exploration, arithmetic-circuit optimization, verification, and software bring-up. Richard Ho, who leads OpenAI's hardware program, said the team optimized the architecture around the serving patterns that matter most for frontier models.
How does the OpenAI Jalapeño architecture reduce latency?
Language-model inference has distinct phases. Prefill processes the incoming prompt and tends to be compute-heavy. Decode generates tokens sequentially and is often constrained by memory bandwidth, while communication between cores or chips can leave expensive compute idle. OpenAI says Jalapeño reduces those delays by keeping model state, including the key-value cache, local and explicitly placed. The system can then activate the needed mix of compute, memory, and networking for each phase without dividing prefill and decode into permanently separate chip pools. SemiAnalysis describes this as a homogeneous architecture designed to remain useful as the ratio of input tokens, cache reads, cache writes, and output tokens changes. The network is part of the design rather than an added transport layer. This co-design aims to place Jalapeño on the throughput-latency Pareto frontier: increasing work per kilowatt without forcing interactive users to accept slower token delivery.
The architecture is designed for multiple model families, not only OpenAI models. The official engineering report shows results for GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Supporting a new model still requires new kernels and model-specific tuning, however. OpenAI says Codex with GPT-Astra helped bring those three open-weight models to high performance in two months. On selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-written versions; that improvement does not describe full-model speed.
What did the InferenceX benchmarks measure?
OpenAI used SemiAnalysis' InferenceX benchmark with nominal 8k input and 1k output tokens under single-token prediction. The published comparisons did not use speculative decoding, multi-token prediction, or separate prefill and decode pools on Jalapeño. Results were normalized with each accelerator's published package power rating: 700 watts for Jalapeño, 1,200 watts for GB200, and 1,400 watts for GB300. OpenAI says Jalapeño's measured sustained power stayed at or below 550 watts during the tested workloads.
Richard Ho told TechCrunch that “Jalapeño can serve more AI work per unit of power, while also returning responses more quickly.” The table supports that claim within the tested operating points: Jalapeño improved both energy-normalized throughput and interactive latency instead of maximizing one by sacrificing the other.
Why do tokens per kilowatt and time between tokens matter?
Tokens per kilowatt measures how effectively an inference system converts a limited power budget into served output. That matters when a data center cannot add grid capacity as quickly as it can install accelerators. Time between tokens, or TBT, captures the delay users feel during generation. A high-throughput system can still deliver a poor interactive experience if each user waits too long between tokens.
Jalapeño's appeal is the reported combination of both metrics. Faster per-user generation can make coding assistants and multi-step agents more responsive, while higher throughput per kilowatt can serve more concurrent demand inside the same facility power envelope. This connection between silicon, memory, and software also echoes how an AI harness changes performance: application speed depends on the complete system, not one theoretical chip number.
What specifications have been disclosed?
OpenAI's report confirms a 700-watt package rating and says sustained consumption was no more than 550 watts during the published workloads. SemiAnalysis says the tested part was the A0 stepping. Its analysis reports that the later B0 stepping is manufactured on TSMC N3P, targets 13.4 PFLOPS of MXFP4 compute, and is expected to improve performance per watt by about 25%. Those B0 figures are specifications and projections, not the measured A0 benchmark result.
SemiAnalysis also reports HBM4 memory projected to deliver approximately 15.4 TB (pending official confirmation)/s of package bandwidth and a fabric designed for a 2,048-accelerator multi-rack domain. That memory emphasis fits the wider effort to solve the AI memory bottleneck, because token generation often waits on weights and cached state rather than arithmetic. OpenAI has not published a complete bill of materials, acquisition price, cloud rental price, independent reliability result, or production cost per token.
How strong is the evidence?
The benchmark has more credibility than a theoretical peak-performance claim because SemiAnalysis visited OpenAI's lab and observed the InferenceX runs. Yet SemiAnalysis' own assessment states that OpenAI supplied all numbers, that it did not run the complete suite, and that no AgentX results were available. AgentX uses long-context, multi-turn workloads that place more pressure on routing, prefix-cache management, KV-cache offload, and the serving framework.
The comparison also spans different product moments. GB200 and GB300 are commercially available Blackwell systems, while Jalapeño remains an engineering sample. SemiAnalysis argues that NVIDIA Vera Rubin, which also uses HBM4 and is beginning to reach customers, will be the more relevant rival when Jalapeño scales. Software maturity will continue changing results on both platforms. A lead measured in August 2026 should therefore be treated as a time-stamped result, not a fixed architectural verdict.
Three other limits matter:
- The 8k/1k test represents a controlled single-turn request, not every ChatGPT or agent workload.
- OpenAI has not disclosed independent production uptime, failure rates, or fleet-scale efficiency.
- Better chip economics do not automatically determine ChatGPT or API pricing, which also reflects models, networking, data centers, software, and commercial strategy.
When will Jalapeño enter production?
OpenAI says deployment inside its own compute infrastructure will begin by the end of 2026 while production qualification and software work continue. Ho told TechCrunch that initial volumes would be very small, with more meaningful deployment in 2027. OpenAI says a second generation is deep in development and a third is taking shape. Broadcom CEO Hock Tan described the program as “just the beginning of a multi-generation roadmap.”
Jalapeño gives OpenAI a first-party inference option, but it does not replace the company's outside suppliers. OpenAI explicitly says it will continue deploying NVIDIA accelerators for training and inference while working across Microsoft, AWS, AMD, Broadcom, Cerebras, CoreWeave, Oracle, SoftBank, and other infrastructure partners. The strategic benefit is choice: OpenAI can route workloads according to capability, latency, power efficiency, availability, and cost. That cost-versus-capability logic is also visible in BriefFlash's GPT-5.6 price-performance analysis.
For users, the practical effects will depend on deployment scale. A faster chip can reduce response delays, support longer agent workflows, and increase capacity during demand spikes. It cannot by itself solve model accuracy, agent reliability, governance, or trust. Jalapeño's next meaningful tests are independent AgentX measurements, production-fleet data, and direct comparisons with Vera Rubin under matched software and workloads.
Key Takeaways
- Jalapeño delivered 1.5–1.9× more peak AI work per watt and 1.7–3.6× lower end-to-end latency than the tested GB200 and GB300 systems.
- The 700-watt inference chip was tested across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T using single-token prediction.
- SemiAnalysis observed the lab runs but says OpenAI supplied the figures; it did not run the full suite, and no long-context, multi-turn AgentX result is available.
- OpenAI plans limited internal deployment by the end of 2026 and a broader ramp in 2027 while continuing to use NVIDIA and other infrastructure partners.
FAQ
What is OpenAI's Jalapeño chip?
Jalapeño is OpenAI's first custom inference accelerator, designed with Broadcom for serving large language models. It is intended to improve token throughput, user latency, and energy efficiency rather than train frontier models.
Is Jalapeño faster than NVIDIA Blackwell?
In OpenAI's early InferenceX tests, Jalapeño led the tested GB200 and GB300 configurations across energy-normalized throughput and latency. The result is limited to specific 8k/1k workloads on engineering silicon, however, and does not establish superiority across every model, serving stack, or production environment.
When will OpenAI deploy Jalapeño?
OpenAI plans to begin limited deployment inside its own infrastructure by the end of 2026. Richard Ho told TechCrunch that volumes would initially be very small, with more significant deployment expected during 2027.
