OpenAI’s Jalapeño Chip Beats Nvidia Superchips on Inference Benchmark

👁 116 reads

OpenAI has announced that its custom-designed Jalapeño chip has outperformed Nvidia’s latest superchips in benchmark tests focused on AI inference workloads. The results, published on Tuesday, show significant improvements in both energy efficiency and speed, marking a major step in OpenAI’s strategy to develop its own specialized hardware for running AI models.

Benchmark Results Show Major Performance Gains

According to the benchmark results published by OpenAI, the Jalapeño chip delivered impressive performance compared to Nvidia’s GB200 and GB300 superchips. The testing showed that Jalapeño achieved 1.5 to 1.9 times more AI work per watt, meaning it can process significantly more artificial intelligence tasks while consuming the same amount of electricity. This energy efficiency is crucial for large-scale AI operations where power consumption directly impacts operating costs and environmental footprint.

The chip also demonstrated 1.7 to 3.6 times lower end-to-end latency compared to the Nvidia superchips. Latency refers to the delay between when a request is made to an AI system and when it delivers a response. Lower latency means faster responses for users interacting with AI applications, which is essential for real-time applications like chatbots, coding assistants, and AI agents that need to respond quickly to user inputs.

The tests were conducted on InferenceX, a specialized benchmarking platform designed to evaluate AI inference workloads. Three different large language models were used in the testing: GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T. These models represent different architectures and sizes, providing a comprehensive view of the chip’s capabilities across various AI workloads.

What Makes Jalapeño Different From Standard AI Chips

Jalapeño is an Application-Specific Integrated Circuit, commonly known as an ASIC. Unlike general-purpose graphics processing units that can handle many different types of computing tasks, an ASIC is designed from the ground up to excel at one specific job. In this case, Jalapeño is built specifically for inference, which means running already-trained AI models to complete tasks or deploy agents. This is different from training, which is the process of initially teaching an AI model by processing massive amounts of data.

OpenAI developed Jalapeño in partnership with Broadcom, a major semiconductor company. According to OpenAI hardware vice president Richard Ho, the chip offers the “best of both worlds” with lower latency and higher throughput. Throughput refers to how much work a chip can process in a given time period. Ho described this combination as unusual because AI systems typically “have to make a trade-off between the two.” Usually, optimizing for speed means sacrificing the ability to handle large volumes of work, and vice versa.

Deployment Timeline and Strategy

OpenAI plans to deploy Jalapeño in “small volumes” before the end of 2026, with production ramping up through 2027. This gradual rollout allows the company to test the chips in real-world conditions and refine their implementation before full-scale deployment. The company is already working on future iterations, with second- and third-generation versions of the chip currently in development.

Importantly, Ho clarified that OpenAI does not intend to retire Nvidia hardware from its infrastructure. Instead, the company is pursuing a hybrid compute strategy that includes “very good partners” alongside its own silicon. This approach allows OpenAI to leverage the strengths of different hardware solutions while reducing dependence on any single supplier and optimizing for specific workloads where custom chips provide the greatest advantage.

Frequently Asked Questions

What is the Jalapeño chip designed to do?

Jalapeño is an Application-Specific Integrated Circuit built in partnership with Broadcom and designed specifically for inference, which means running trained AI models to complete tasks or deploy agents. It is not designed for training AI models, but rather for using already-trained models in production environments.

How much better is Jalapeño compared to Nvidia chips?

In benchmark tests against Nvidia’s GB200 and GB300 superchips, Jalapeño delivered 1.5 to 1.9 times more AI work per watt and achieved 1.7 to 3.6 times lower end-to-end latency across three different AI models tested on the InferenceX platform.

When will OpenAI start using Jalapeño chips?

OpenAI plans to deploy Jalapeño in small volumes before the end of 2026, then ramp up production through 2027. The company is already developing second- and third-generation versions of the chip while maintaining its partnership with Nvidia for other computing needs.

// read moreFrench startup Kog raises seed funding to speed LLM inference on GPUsAI ToolsApple launches M6 and M5 Ultra chips in new Mac Mini and Mac StudioAI ToolsNvidia, Stripe Lead $26 Billion Buying Spree Into Open-Weight AIMagazine

Kinetyra