3 articles
Inference
All posts tagged with #Inference
DeepSeek Open-Sourced Its Inference Stack. Inference Prices Won't Survive It.
DeepSeek released DeepInfer, the open-source inference framework behind V4.1-Flash. Here's why free production-grade serving code pulls the whole market's prices toward the floor — and the US policy catch nobody should ignore.
One argsort, up to 17% faster: the free MoE inference speedup
Sorting prompts by domain before batch generation made stock vLLM up to 17% faster on MoE models, with no model or kernel changes. Here is why it works and how to apply it.
750 Tokens a Second: Speed Is Deciding Which AI Agents Survive
OpenAI and Cerebras previewed an 11x-faster API tier this week, and Google and Nvidia shipped speed-focused releases the same week. Why latency now decides which AI agents are usable in production, and how to evaluate it before your next tool purchase.