We just hit 300 members today—woke up to 100+ new people joining overnight. Come join these top DIY hackers using extreme optimization to make old hardware run Qwen3.8-Flash and GLM5.3-Flash. 🚀xitter.cf/i/chat/group_join/g209…

4× V100 96GB VRAM test on ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S: Strata is far ahead of llama.cpp+DFlash—9.05× prefill, 1.67× decode. Can’t wait for Strata + DFlash + this model. And the quant quality is f***ing good—on par with Q6 so far. Gonna run more tasks on it.

OMG, 4000 TPS prefill. I’ve never seen such speed on my V100 rig.😍

Qwen3.8-Flash IQ4 + DFlash2 + llama.cpp: decode up to 28% faster 🚀

Bad news, NVIDIA CMP 170HX price curve is insane: $200 last year, $1,500 last month, now over $3,000—still less than half an RTX 5090. Specs are wild: A100-class performance, 64–80GB VRAM, bandwidth close to an RTX 5090, SM80 compute, and CUDA 13 support. If you don’t want to tinker with V100s, an unlocked 170HX is the no-brainer combo of value and convenience. Good news, supply is still plentiful.

🚀 Local inference engines worth following: @coldniko (Strata/Qwen3.8-Flash), @MiaAI_lab (DGX Spark), @inco_ai (Splash+DFlash/Mac), @Andy_ShuoYang (FreeToken/NVFP4), @PeasantSmith (Maya/GLM5.3-Flash).

Must have @coldniko 😎🔥

In China, a Mac Studio M5 Ultra with 512GB costs about the same as BYD’s hottest new SUV or sedan. Imagine that: the price of a car almost any family can afford now buys you the ability to deploy world-class SOTA models at home. A few years ago, that picture would have been unbelievable.

Good news: 512K context with DeepSeek-V4.1-Flash 1017 IQ1 runs easily on my 192GB RAM + 96GB VRAM. Bad news: only 11 TPS decode. Oh boy—llama.cpp 😅

This great man deserves to be followed—come on, guys.

No need to scramble actually—DDR3 is more than enough, and there’s plenty to go around. tomshardware.com/video-games…

Last year I never thought I could cancel all my online SOTA subscriptions. This year, Strata + Qwen made it happen. Want to do it with a $1000~1,500 DIY build? Factor in local power costs. People ask about draw—mine’s under $1/day when system on, since it’s not always at full load. What’s electricity like where you are?

Tempting me to upgrade to 256GB RAM 😭 Good news: even IQ1 hits 88% top-1—already beating Qwen3.8-Flash Q4—and IQ2 reaches 94%, close to Qwen3.8-flash Q6. Doesn’t feel out of practice anymore. Bad news: you need 128GB of RAM + VRAM just to start, and there’s no Strata-style optimization for older GPUs. The real-world barrier is still high.

Need more GPUs? This is what your future eGPU dock looks like 🤣

Suddenly there are tons of inference engines optimized for a specific model—or even a specific quant—popping up everywhere. Good news: faster iteration and better performance. Bad news: my disk space is running out 😅

I don't know what's in my head. Q4 and Q8 were only 8GB apart, so I downloaded it without thinking. My rig can’t handle it at all—but I still can’t bring myself to delete it 😂

Make Local LLMs Great ever. 🚀