AMD GPUs take prefill, CS-3 takes decoding in Cerebras' split inference architecture

Cerebras has taken what has long been a research topic among the major cloud providers — disaggregated inference — into its own production. The company announced on November 17, 2026, that its inference service now splits the work between…

File photo: AMD Athlon XP 1700+ processor
File photo: Matthieu Riegler English : This photo has been taken by Matthieu Riegler ( User:Kyro ) and released under the licenses stated below. You are free to use it for any purpose as long as you credit me as author, Wikimedia Commons as site and follow the terms of the licenses. Could you be kind enough to leave me a message on this page to inform me about your use of this picture. Example : © Matthieu Riegler , CC-BY Français : Cette photo a été prise par Matthieu Riegler ( User:Kyro ) et placée sous les licences ci-dessous. Vous êtes libre de la réutiliser, pour n'importe quelle utilisation, tant que vous me citez en tant qu'auteur, Wikimedia Commons en tant que site et suivez les instructions des licences. Pourriez-vous avoir l'amabilité de me laisser un message sur cette page pour m'informer de votre utilisation de cette image. Exemple : © Matthieu Riegler , CC-BY This file is licensed under the Creative Commons Attribution 3.0 Unported license. Attribution: Matthieu Riegler, CC-by You are free: to share – to copy, distribute and transmit the work to remix – to adapt the work Under the following conditions: attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. https://creativecommons.org/licenses/by/3.0 CC BY 3.0 Creative Commons Attribution 3.0 true true Vous souhaitez faire apparaitre cette photo sans crédit ? Contactez moi ! If you want to use the photo without credit, please contact me ., CC BY 3.0 · source

AMD GPUs take prefill, CS-3 takes decoding in Cerebras' split inference architecture

Cerebras has taken what has long been a research topic among the major cloud providers — disaggregated inference — into its own production. The company announced on November 17, 2026, that its inference service now splits the work between two entirely different machine types: AMD Instinct GPUs process the user's prompt, while Cerebras' CS-3 systems built on wafer-scale chips generate the response. According to the company's own measurement, the combination delivered five times higher throughput per megawatt than a CS-3-only configuration, with no loss in token speed (Cerebras blog).

The figure comes from a measurement dated November 14, 2026, in Cerebras Inference Cloud, made on a shared GPT-OSS-120B endpoint and normalized against a so-called aggregated-inference baseline. It is thus a self-reported measurement, not an independent benchmark, and the details of measurement conditions, workload, and hardware configuration have not been published in full.

Why split inference?

Running a language model consists of two phases with very different characteristics. In the prefill phase, the entire prompt is read in and the model builds an internal representation of it — heavy computation that parallelizes well. In the decode phase, tokens are generated one at a time, and each decision depends on reading the accumulated state (including the key-value cache, KV cache) quickly from memory.

Cerebras' argument is that the decode phase is limited by memory bandwidth, not compute power. That is where the company's wafer-scale design fits prefill poorly but decode well: the CS-3 wafer has, according to the company, "tens of gigabytes" of SRAM and "tens of petabytes per second" of memory bandwidth on a single chip — stated as 350 times the SRAM capacity and 5,400 times the memory bandwidth of a typical NVIDIA GB300 GPU. These figures, too, are the company's own claims, not verified facts.

The logic of the split is therefore simple: give the prefill work — which GPUs excel at — to AMD Instinct GPUs, and let the full CS-3 capacity go to token generation, where its enormous SRAM and bandwidth resources are actually utilized. The company says the move was driven by demand: the development of disaggregated inference began, according to the blog post, "to meet demand."

The plumbing: KV Portal

The technical intermediary is KV Portal, software that transfers intermediate model state — including the KV cache — from the prefill engine to the decode engine over RDMA. RDMA (remote direct memory access) allows direct memory access between machines with low latency, which is critical now that prefill and decode happen on physically separate systems.

For partners, Cerebras offers a reference architecture and an end-to-end software stack for disaggregated inference, as well as a decode-only SDK whereby a partner can let Cerebras systems handle decoding alone. The details around this SDK path are limited in the published material.

Ecosystem and rollout plan

Four partners are named with early access: Amazon Web Services, Fireworks AI, Gimlet Labs, and General Compute, as reported by LavX News with reference to the launch blog. Cerebras says access will expand to more partners.

The rollout plan is ambitious: the company plans to roll out disaggregated inference "broadly" across hundreds of megawatts in Cerebras Inference Cloud and in partner deployments by the end of 2027. This is the company's own plan, not a verified outcome, and hundreds of megawatts is a large-scale scaling ambition for a company of Cerebras' size.

The context: efficiency becomes the bottleneck

Behind the interest in this architecture is a rough gap between demand and capacity. Goldman Sachs Research predicts, according to LavX News — as the figures are cited circulating around the launch — that AI inference demand will reach 24 times 2026 levels by 2030, while data-center capacity grows only 2.2 times over the same period. Note that these figures rest on secondary sources: the underlying Goldman Sachs report is not available in the basis on which this article is built, and the numbers are relayed through LavX News' retelling and Cerebras' own context.

If the forecasts have the right direction, tokens per megawatt becomes one of the most important metrics for inference infrastructure. That is exactly where Cerebras places its main message.

What remains open questions

  • Verification. The 5x-per-megawatt figure is a self-reported measurement from one shared endpoint of one model (GPT-OSS-120B), normalized against the company's own baseline. No independent benchmark is yet available.
  • Generalizability. How the architecture performs on other models, other prompt/output-length mixes, and other configurations is not documented. In principle, the result could be favorably influenced by the composition of the workload.
  • Methodology. The measurement conditions on November 14 — hardware setup, concurrency, traffic pattern — are not fully described in the published material.
  • Rollout. Whether the partner deployments actually materialize by the end of 2027, and at what scale, remains to be seen.
  • SDK details. The decode-only path for partners is described only to a limited extent in the source material.

What it means

Regardless of whether the 5x figure holds up under independent scrutiny, the architectural message is clear: Cerebras is positioning its wafer-scale chips not as a GPU replacement, but as a specialized decode engine in a hybrid architecture where GPUs do what they do best. It is a pragmatic turn for a company that has historically marketed the CS-3 as a complete alternative compute platform — and an example of inference infrastructure increasingly specializing by phase, not by model.

AIMag.no
AIMag.no
The AIMag.no editorial team covers artificial intelligence, tools, research, and regulation.

Get the best of AI MAG in your inbox

News, analysis, and ideas at the intersection of AI and society.