OpenAI Found a Crack in Nvidia’s AI Fortress

Nvidia has built one of the strongest competitive positions the technology industry has ever seen. It dominates the market for artificial intelligence accelerators, owns the CUDA software ecosystem that developers have spent years learning and has expanded beyond individual GPUs into networking, rack-scale systems and increasingly complete AI infrastructure.

OpenAI’s first custom processor is not about to bring that fortress crashing down. It cannot train frontier models, has not reached volume production and has yet to prove itself across years of real-world operation.

But OpenAI may have found the first crack in the wall.

The company has released detailed performance results for Jalapeño, the inference processor it developed with Broadcom. According to OpenAI’s benchmark disclosure, Jalapeño delivered between 1.5 and 1.9 times more AI work per watt at peak throughput than Nvidia GB200 and GB300 comparison systems. It also produced between 1.7 and 3.6 times lower end-to-end latency.

Those results were recorded across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. The inclusion of three public model families is important because it makes Jalapeño harder to dismiss as an ASIC narrowly optimized around one proprietary OpenAI model.

Jalapeño is not a general-purpose GPU, however. It is an inference processor built to run models after they have been trained. It does not challenge Nvidia’s position in the massive clusters used to create frontier models. What it does challenge is the assumption that Nvidia GPUs will continue to perform nearly all the valuable work across the AI lifecycle.

Inference may be where that assumption first breaks.

From Fast Tapeout to Competitive Silicon

When OpenAI and Broadcom introduced Jalapeño in June, much of the attention centered on how quickly it had been developed. OpenAI said the chip moved from its initial design to manufacturing tapeout in nine months, a remarkably compressed schedule for a high-performance accelerator.

In his June Futurum analysis, “Jalapeño in Nine Months: Did AI Just Break Chip Design Timelines?”, Brendan Burke correctly identified the schedule as the initial breakthrough.

He also supplied the necessary caution. A tapeout is not a finished product. Validation, yields, packaging, system integration and software development still separate an initial design from production infrastructure. At the time, Burke argued that OpenAI’s claimed performance advantage should remain unverified until the company released more detailed results.

Those results have now arrived.

Across the three tested models, OpenAI says Jalapeño produced 1.5 to 1.9 times more work per watt at peak throughput. For highly interactive workloads, it reported between 2.1 and 4.1 times higher performance.

Jalapeño is rated at 700 watts, although OpenAI says sustained consumption remained at or below 550 watts during the tested workloads. The Nvidia GB200 and GB300 chips in the comparisons carry published power ratings of 1,200 and 1,400 watts, respectively.

OpenAI also published some spectacular comparisons showing 53.7, 56.1 and even 104.3 times more throughput at particular matched time-between-token operating points. Those numbers will attract plenty of attention, but they reflect specific points along the performance curve. The broader performance-per-watt results provide the more useful basis for evaluating the architecture.

Even those more restrained numbers are impressive for first-generation silicon.

The Benchmarks Need Context

OpenAI’s results deserve to be taken seriously, but not accepted without qualification.

The tests used InferenceX, a public benchmark developed by SemiAnalysis. SemiAnalysis representatives observed the runs in OpenAI’s lab, but their assessment notes that OpenAI supplied the numbers. They did not run the entire InferenceX suite and have not yet seen results from AgentX, which is designed to reflect longer-context, multi-turn agentic workloads more closely.

That becomes increasingly relevant as inference shifts from answering individual prompts to operating agents that maintain context and execute many sequential steps. An architecture can perform exceptionally well on an 8,000-token input and 1,000-token output test without necessarily sustaining the same advantage under more complicated production loads.

There is also a generational mismatch to consider. Jalapeño uses HBM4 and is expected to begin limited deployment late this year before ramping during 2027. OpenAI primarily compared it with Nvidia’s commercially deployed Blackwell systems. Nvidia’s HBM4-based Rubin generation will be a more contemporary comparison by the time Jalapeño begins operating at meaningful scale.

Engineering samples in a controlled environment are not a production fleet. OpenAI and Broadcom still must qualify the systems, establish acceptable yields, secure advanced packaging capacity, mature the software and demonstrate that Jalapeño can run reliably across thousands of machines.

None of those caveats make the results meaningless. They define what has and has not been proved.

OpenAI has not proved that Jalapeño is superior to everything Nvidia will offer when it reaches volume production. It has shown that a first-generation custom inference processor can be competitive with Nvidia’s current commercially available systems across multiple major model families.

That is a substantial accomplishment.

Inference Is the Vulnerable Wall

Training and inference are both described as AI computing, but they are becoming increasingly different semiconductor markets.

Training frontier models demands maximum flexibility. Model architectures continue to evolve, clusters span enormous numbers of processors and developers rely on mature libraries and tools to keep the entire system running. Nvidia’s GPUs, CUDA ecosystem, networking and rack-scale engineering provide a combination that competitors have found exceedingly difficult to match.

Inference creates a different set of economics.

Once an AI provider reaches sufficient scale, its serving workloads become large, repetitive and relatively predictable. It knows how its models use memory, how they move between prefill and decode, where KV caches should reside and which communication bottlenecks create latency.

That knowledge can be embedded in silicon.

OpenAI designed Jalapeño around the work it expects modern language models and interactive agents to perform. The architecture attempts to minimize data movement, keep model state local and coordinate compute, memory and networking across inference phases.

At OpenAI’s scale, performance per watt is more than a benchmark. AI infrastructure is increasingly constrained by access to electricity, data center capacity, networking and advanced packaging. If a custom processor can complete 50% more useful work from the same power envelope, the result affects both the cost of delivering a response and the amount of revenue that can be produced from each megawatt of infrastructure.

OpenAI does not need Jalapeño to replace Nvidia across its entire compute estate. It needs the processor to absorb enough inference demand to improve its economics.

That could reduce the cost of operating ChatGPT and the OpenAI API, increase the capacity available for agents, improve interactive response times and lessen OpenAI’s dependence on Nvidia’s supply. It would also give OpenAI leverage when negotiating future purchases and infrastructure commitments.

The number to watch is not one benchmark score. It is the percentage of OpenAI’s total inference workload that Jalapeño and its successors eventually handle.

OpenAI has not disclosed that target. Until it does, any estimate would be speculation. But even a minority of OpenAI’s rapidly growing inference demand would represent a large pool of processors that otherwise might have come from Nvidia.

The Stacks Are Colliding

Jalapeño also reveals a larger strategic collision.

Nvidia began with accelerators and has steadily moved up the stack. It added networking, systems, software, cloud services and models. Its objective is no longer simply to sell chips. Nvidia wants to define and capture as much of the AI computing platform as possible.

OpenAI is traveling in the opposite direction. It began with models and applications. It is now moving down into chips, memory architecture, networking, systems, data centers and energy.

OpenAI explicitly describes Jalapeño as evidence of a full-stack advantage spanning models, products, serving software, silicon, memory, networking and rack-scale infrastructure. That language could just as easily have come from Nvidia.

Both companies understand the danger of remaining confined to one layer.

Nvidia does not want its GPUs reduced to interchangeable components inside platforms controlled by model providers and hyperscalers. OpenAI does not want the economics of its products permanently determined by the margins, roadmaps and supply constraints of its infrastructure providers.

They are approaching the same strategic ground from opposite ends of the stack.

This does not mean OpenAI built Jalapeño by itself. Broadcom’s contribution was essential. OpenAI supplied detailed knowledge of its workloads and the architecture it wanted, but Broadcom brought years of custom-silicon implementation, advanced packaging and high-performance networking experience.

Broadcom’s packaging capabilities and Tomahawk networking provide much of the industrial foundation required to turn an architecture into a manufacturable chip and then make thousands of accelerators behave like one system. Jalapeño demonstrates the growing capability of the custom-silicon ecosystem surrounding Nvidia. It does not demonstrate that every well-funded AI startup can suddenly design a competitive processor in nine months.

AI Designs the Chip That Runs AI

The development process may prove almost as consequential as the finished processor.

OpenAI says its models helped the engineering team explore implementations, accelerate verification and measurement loops and optimize arithmetic circuits. This was not an AI system autonomously inventing a chip and handing a completed design to a foundry. It was AI multiplying the work of experienced engineers and compressing time-consuming optimization cycles.

The process continued after tapeout. OpenAI used Codex with GPT-Astra to bring three additional open-weight models to high performance on Jalapeño in approximately two months. For selected GPT-OSS attention and mixture-of-experts blocks, OpenAI says AI-generated implementations ran between 1.5 and 1.8 times faster than existing implementations written by human experts.

Those figures apply to selected blocks, not complete models. Still, they point toward a powerful development loop.

Better AI can accelerate the design and programming of better chips. Those chips can then run AI faster and more efficiently. The resulting models can assist with the next generation of silicon.

OpenAI says its second-generation processor is already deep in development and its third is beginning to take shape. Those chips will reveal whether the nine-month timeline and performance gains represent a repeatable advantage or an exceptional first effort assembled through an unusual combination of talent, capital and partners.

The Fortress Still Stands

Nvidia’s fortress remains formidable. OpenAI continues to need Nvidia and other suppliers for training and will continue deploying their accelerators for inference. CUDA, NVLink, Spectrum-X and Nvidia’s rack-scale systems cannot be displaced by one benchmark or one chip that has not yet reached volume production.

But fortresses rarely fall in one dramatic assault. Their vulnerability first becomes visible when someone proves that a portion of the territory can be taken.

OpenAI has now shown that a first-generation custom processor can compete with Nvidia’s currently deployed systems on inference speed and performance per watt. If Jalapeño and its successors begin absorbing a meaningful portion of OpenAI’s inference demand, other hyperscalers and frontier-model companies will reach the same conclusion.

Nvidia may remain indispensable for some of the most valuable AI workloads without remaining indispensable for all of them.

The question is no longer whether OpenAI can design a working AI chip. It has. The question is how much Nvidia-powered inference OpenAI can move onto its own silicon—and how quickly the rest of the AI industry follows it through the first crack in the wall.