DeepSeek infer is the cloud version of DwarfStar, conceptually. Vertical inference engine for one model and specific hardware. Good move. And well, a lesson to the ones that criticized this approach as well.

Oct 20, 2026 · 4:30 PM UTC

24
26
1
797
29,963
RelevantRecentLikes
when the next deepseek comes out, how much of a vertical engine like dwarfstar can you reuse, or is it mostly a rewrite each time?
1
3
2,648
Between 4.0 and 4.1 it was easy, till something completely new appears in the attention side, all the rest can be kinda reused.
1
25
6,305
How much of DeepSeek Infer can be ported into DwarfStar context? Are there lessons and speedups available we could recycle to the local inference context?
1
3
2,234
SM100 is a peculiar architecture compared to SM121, much more powerful. Something can be done, but not much, also Infer is specially suited for batching while local is often single session.
1
8
1,715
I’m drawing on deepinfer’s DSpark confidence-scheduled varlen verification approach and trying to make DSpark produce positive gains in DwarfStar under SSD streaming mode. Currently, on a 128GB M5 Max, decode is stable at 21–23 t/s. After a few more tests, I plan to submit a PR.
113
specialized inference is underrated..tuning kernels, batching nd memory management for one model + hardware can improve tok/s and cost more than people expect. it's also why the same weights can feel completely different across providers in kilo, the serving stack matters too.
15
Vertical runtimes trade away abstraction layers to eliminate the dispatcher tax entirely. When you are serving millions of tokens at scale, hardcoding memory layouts and kernel dispatches directly to the hardware pays for itself on day one.
1
195
Spot on. The era of bloated, "one-size-fits-all" inference frameworks is fading. Extreme vertical integration (specific model + specific hardware) is the only way to survive the compute crunch. DeepSeek just proved that hardcore infra engineering beats brute-force scaling.
98
sm100 only means even a 5090 isn't on the list, so the consumer side stays wide open for dwarfstar-style engines. one model, one box, tuned for a single user
419
General-purpose runtimes carry too much layout overhead for single-model vertical setups anyway.
490
even deepseek's vertical engine started life as a general one. deepinfer's readme says it was originally built primarily on vllm, and it now supports exactly one model, on nvidia sm100 and huawei ascend 950 github.com/deepseek-ai/DeepI…

GitHub - deepseek-ai/DeepInfer: DeepInfer: High-performance, large-scale inference framework for...

DeepInfer: High-performance, large-scale inference framework for DeepSeek models - deepseek-ai/DeepInfer

github.com
2
708
Wen | 阿栗AI
@xiaoshan0413Oct 20
垂直即正义
400
I owe so much to dwarfstar🙏i know ur an atheist but I pray for your continued success!
7
1,233
it's a good approach, but back of the enveloppe shows we can match speed
1
2
1,069
So many critics...advise check history with AI checks as somehow it helps to remind people is option now. Their history tells why so often wrong. As is typical of many...not all...but many.
923
Se puede decir que, DeepSeek infer copió a DwarfStar 😅 o solo confirmar que las grandes mentes pueden coincidir ❤️
225
I bet this is how it works in big AI companies. They might even have an entire team working on super optimised inference engine for each model release.
764
Io non capisco che si intende con versione cloud, quando usi servizi tipo chatgpt o claude non lavori in cloud? Cosa cambia tra quello che fa DS e gli altri? E con tutto il loro hardware perché devono quantizzare il modello come da DwarfStar?
1
198
Arriveremo al momento che ogni modello scriverà il suo proprio runtime. Bisognerebbe fare qualche esperimento!
254
this feeling if you make decisions and all are wondering and than later all agree with that. love it also.
491
I was waiting for this post lol great Release
287
Load more replies