• Cerebras says its new CS-4 runs AI inference up to 30 times faster than GPUs, but the figure measures one user’s wait, not a rack’s total output.
• The claim is aimed at agents, which run their steps in sequence and pay for every second of delay.

Cerebras Systems, an AI chip startup that is seen as a potential challenger to Nvidia in the AI infrastructure race, recently introduced its latest AI inference system built from three wafer-scale processors. The system, CS-4, according to the company, produces more than 4,400 tokens per second per user on GPT-OSS-120B, an open-weight model with 120 billion parameters. 

Cerebras describes the CS-4 as the fastest AI accelerator in the industry. It is the first system built on the company’s next-generation Nexus rack-scale platform architecture. The system is almost twice as fast as the CS-3, and up to 30 times faster than GPU solutions on tokens per second per user, given identical prompts.

The design is unusual. Rather than cut a silicon wafer into hundreds of individual chips and connect them on a board, Cerebras keeps the wafer intact and runs it as one processor, with 44GB of memory built onto the wafer itself. That matters for the speed claim, because moving data between a chip and its memory is what slows AI inference down, and there is far less distance to travel here.

Cerebras puts the figure at 43.2 petabytes per second of memory bandwidth per wafer, double the previous generation, and says in the release that memory bandwidth is the determining factor in driving speed and throughput.

For context, tokens per second per user measures how quickly words arrive for one person waiting on one answer. Throughput measures how many tokens the whole system produces for everyone using it at once, and the two usually pull against each other. GPU clusters group incoming requests into batches, which keeps the hardware busy and the cost per token low, but every request in the batch waits its turn. Cerebras is built for the opposite priority.

Put in human terms, 4,400 tokens a second is roughly 3,300 words. A fast reader manages five. Somewhere well below Cerebras territory, the text is already arriving faster than anyone can absorb it, and the extra speed goes unseen.

AI inference built for agents, not readers

That is the case Cerebras is making. “Being 30 times faster doesn’t just make a response feel fast. It gives an agentic system room for more than an order of magnitude as much reasoning, verification, or tool use in the same wall-clock time,” the company’s CTO and co-founder Sean Lie said in the release. “That’s the difference CS-4 makes for real production workloads.”

So the CS-4 is a bet on which workload wins. If enterprise AI stays largely conversational, per-user speed is a line on a datasheet. If it shifts to agents running long chains unattended, it becomes the thing everything else waits on.

Here’s how the logic works. Say an agent is asked to reconcile an invoice. It reads the invoice, looks up the purchase order, compares the two, finds a discrepancy, then queries a third system to explain it. Each step has to wait for the one before it comes back. The delays stack rather than overlap.

That is a different shape of demand from a hundred people each asking one question, which is the workload most inference infrastructure was sized for. Speed for a single request matters far more to the agent making forty calls in a row than to any one of those hundred people.

In comparing the key specifications of CS-3 vs. CS-4, Cerebras noted that compute rises from 125 to 750 petaFLOPS across that comparison, which is six times the output from three times the silicon. The efficiency figure shifts baseline too. Cerebras says the CS-4 delivers up to 10 times more throughput per watt, but that is against its own CS-3, not against GPUs.

The ‘30 times faster than GPU solutions’ comparison, however, names no specific GPU. Cerebras does not disclose the vendor, the model or the serving configuration of the system it measured against, and the release’s own footnote says actual throughput varies by model architecture, context length, precision and serving configuration. 

The number that matters to whoever pays the power bill

Throughput per watt is the figure with the most immediate use for anyone buying AI inference capacity because power rather than land or capital is now the constraint on where data centres get built and how large they get. Cerebras attributes part of the gain to moving power conversion around 100 times closer to the processor, from roughly 50 millimetres on conventional GPU boards to about 0.5 millimetres, which it says nearly eliminates board-level power loss.

It matters for an operator working inside a fixed megawatt allocation because tokens per watt sets how much sellable capacity that allocation represents. That sum stands on its own, whatever the comparison with Nvidia turns out to be.

Further down the release is the detail that says most about where Cerebras sees itself. It describes the CS-4’s programmable I/O subsystem as particularly beneficial for disaggregated inference, naming AMD Helios and AWS Trainium as ecosystem partners.

Inference splits into two jobs. Prefill is reading the question. Decode is writing the answer. In a disaggregated setup, one system handles the first and hands off to another for the second, and Cerebras is positioning itself for the second. Not a replacement for the GPU cluster, then, but a piece bolted onto one. First shipments begin this quarter.

Want to experience the full spectrum of enterprise technology innovation? Join TechEx in Amsterdam, California, and London. Covering AI, Big Data, Cyber Security, IoT, Digital Transformation, Intelligent Automation, Edge Computing, and Data Centres, TechEx brings together global leaders to share real-world use cases and in-depth insights. Click here for more information.

TechHQ is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.