For two years the AI tool conversation ran on two specs: how smart is it, and what does it cost. A third spec crashed the party this week, and almost nobody’s roundup covers it.
OpenAI and Cerebras previewed Ultrafast, an API tier that runs GPT-5.6 Sol on Cerebras chips instead of OpenAI’s own hardware. It pushes up to 750 output tokens per second. For scale: Artificial Analysis measures standard GPT-5.6 Sol at about 65 tokens per second. That’s roughly 11x faster, and it changes what you can build.
There’s no pricing yet, no general availability date, and it’s a limited preview behind a waitlist. That part is boring. The interesting part is what the number does.
The engineering trick, in one paragraph
Cerebras keeps the model’s weights in 44GB of on-chip SRAM. GPUs constantly shuttle data between memory and compute, and those trips are the bottleneck. Skip the shuttle and the tokens pour out. That’s the whole idea, and it’s the reason the speedup is so lopsided compared to usual generation-over-generation gains.
End-to-end numbers beat throughput bragging
Raw tokens per second is a lab number. What convinced me in The Batch’s coverage (DeepLearning.AI, Aug 29, 2026) was the task-level data.
On six quality-matched GDPVal tasks, Cerebras measured 83 seconds per task on Ultrafast versus 7.7 minutes for standard Sol. A 5.6x speedup while holding quality flat. And Ultrafast ground through all 2,500 Humanity’s Last Exam questions in 11 hours 11 minutes. Claude Fable 5 needed 78 hours 27 minutes for the same set.
Stop and sit with that second number. Same workload, same questions. One finishes before your Monday standup, the other is still running Thursday.
Speed stopped being a luxury spec
I think most coverage misses the point here. Speed isn’t about impatience. It decides which product categories physically work.
Voice is the obvious one. Talk to an assistant that pauses longer than about a second and your brain files it under broken, even when the answer is brilliant. At 65 tok/s, a spoken paragraph takes an awkward while to generate. At 750, conversation keeps its rhythm.
Always-on agents have the same problem from the other direction. An agent that watches dashboards or inboxes for threats has a window to act in. If the analysis pass takes a minute, the window may already be closed. Monitoring that finishes in seconds is the difference between an alert that arrives in time and a postmortem.
Developer agents are the third case, and the subtlest. Nobody queues a task, stares at the spinner, and waits anymore. They tab away, start something else, and pay a context-switch tax every time they come back. An agent that finishes before you lose the thread earns a permanent spot in the workflow. A slow one teaches the team to stop invoking it.
The same week made it a trend, not a one-off
Ultrafast is the headline, but two other releases landed with speed as the story, which is how you know the battleground is real.
Google shipped Gemini 3.7 Flash. Artificial Analysis measured it at 330 output tokens per second with 13.2 seconds to first token, and its Intelligence Index score rose to 56 from 52. Fast and smarter at the same time, which undercuts the old assumption that you trade one for the other.
Nvidia released Nemotron 3.5 Lightning at 302 measured tokens per second, claiming 30% faster agentic task completion. The more interesting release from Nvidia, though, is NeMo Switchyard: an open-source routing library that sends each step of an agentic workflow to whichever model fits best on speed, quality, or cost. Nvidia says routing this way cut task-completion cost to about a third of running Claude Opus 4.8 alone.
Three companies, one week, all betting on the same spec. That’s not a coincidence. That’s the market noticing latency is where the next round of differentiation lives.
What to actually do with this
If you’re evaluating AI tools for a marketing stack or any production use, start tracking speed alongside quality and cost. Concretely:
- Time the tasks, not the demos. Vendor demos pre-load the impressive path. Run your own typical workload end to end and measure wall-clock time, including first-token latency if it’s conversational.
- Ask vendors for both numbers. Tokens per second and seconds to first token. A model can stream fast but leave you hanging for 13 seconds before the first word, and for voice or chat front-ends, that first token is the whole experience.
- Match speed to the job. Research and drafts tolerate a slow model. Voice interfaces, live monitoring, and in-flow assistance don’t. Write the latency budget down before you shortlist tools, the same way you’d write a budget for dollars.
- Watch routing. If your workflow has many steps, a router like Switchyard that mixes fast cheap models with slow smart ones may beat any single model on both cost and total time. This is where agentic economics are heading.
- Don’t buy on throughput alone. Ultrafast’s GDPVal numbers matter because quality held flat while speed jumped 5.6x. A fast model that flubs the task is just an expensive way to be wrong sooner.
The asterisk on all of this
Ultrafast has no pricing and no GA date. The economics of SRAM-resident inference at scale are unproven, and a waitlist preview is a promise, not a product. If your roadmap depends on it, plan for the timeline to slip.
But the direction is set. Quality got good enough. Cost keeps falling. Speed is the spec that decides whether the agent on the other end of your product feels like a colleague or a loading bar. The teams that put latency requirements into tool selection now, the same way they already put in cost caps, are betting on the right dial.
The fastest model in the world doesn’t matter. The fastest one that’s actually in your workflow does.
