TechMD retweeted
Wow 😲
Cloudflare's Clef is now #1 on HuggingFace!!
Cloudflare's Clef beat Jev in just two weeks
How: "frozen" Qwen does one prefill pass, a tiny schema head scores every answer option in parallel, with no text generated at all.
That's why it's 4x faster than Jev at 2x the accuracy on some benchmarks.
And it's open source!
Link to HF:
huggingface.co/Cloudflare/cl…
TechMD retweeted
Pagoda's benchmark but Matrix style!
- 7 online models
- 2 local models (more to be added, suggestions are welcome!)
Which one do your prefer?
TechMD retweeted
EXLR8 is cooking in CuteAFD
V4 release will support GLM 5.3
Finally got the prefill to beat standard EXL3 K4!!!
This is a Spark TP6 result with dense BF16 weights for the non expert part where that core runs on 2xRTX6000
The key is 2824 EXL3 vs 2869 EXLR8… winning by a hair 😂
@mmastrac did you manage to find a way to decimate classic EXL3 on prefill with exlr8?
TechMD retweeted
今日ウエイト公開されたこの子をダウンロードしてるよ！！Ablitratedにする。
Deepseek v4.1 flash 1017はThinkingの暴走がなかなか抑えられなくて、大規模なファインチューニングが必要そうだったので一旦保留する。
huggingface.co/MiniMaxAI/Min…
TechMD retweeted
MiniMax-M3.1-Flash TP4 on 4× DGX Spark, day 0 🐇
• Prose decode: 39.7 tok/s at c1
• Structured decode: 62.2 tok/s at c1
• Aggregate: 166 tok/s prose, 261 tok/s structured at c16
• speedbench quick: 38.4 tok/s across 30 prompts
• 262k context, ~1.37M-token FP8 KV cache
• MiniMax's own NVFP4 experts + MXFP8 weights, bundled DSpark draft
• Runs on GB10 (sm_121): FP8 KV, Triton sparse attention, Marlin experts (the NVFP4 KV cache is SM100-only)
First measured bring-up. Speed tuning and quality gates are next.
Full recipe, measurements and credits:
github.com/knapcio/MiniMax-M…
TechMD retweeted
STOP STOP STOP STOP!
You need to try my BlackFold TP2 DeepSeek V4.1 1017 build. Uncensored on TensorFold
YOU actually need to try it right NOW.
Look at what happens when you push it to xhigh reasoning:
🔥 90/100 on tool eval bench (173/192)
⚡ ~70 tokens/sec avg coding speed locally
Huge shoutout to @ivanfioravanti for putting it through its paces and showing what this beast can do outside my own test environment.
If you want absolute raw performance with near-zero refusals running locally on your hardware, stop scrolling.
Model and setup links below. Run it, push it, and show me what you build 👇
DeepSeek-v4.1-Flash-1017-heretic-tf correction! BlackFold test below was with:
--reasoning-effort none.
With xhigh it got 90/100. 173/192 points!
Still Adequate as rating, but mainly for safety, that clearly is irrelevant for this model and Structured Output where it seems Main cause: TensorFold rejects response_format engine rejects both json_schema and json_object. @Blackwellboy is looking into this.
I'm using it for coding right now and it keeps ~70 t/s as average! 🔥
TechMD retweeted
DeepSeek-V4.1-Flash EXL3-EAQ v1.1 is out 🧪
What's different vs v1?
Sparse attention uses an indexer to pick which past tokens matter. v1.1 keeps that whole path in full BF16 instead of quantizing it (the indexer, shared query projection, and compressor).
When fed the original model's inputs, v1.1 makes the exact same picks 100% of the time past 16k (403,560 / 403,560), vs 94.95% on v1.
In a full quantized run, both land near 73%. That's because the inputs reaching the indexer have already shifted from quantizing the rest of the model, so a BF16 indexer alone can't fix that gap.
v1.1 weights just ~0.15 GB more than v1.
Please note that this is experimental ‼️
Get it here:
huggingface.co/Mia-AiLab/Dee…
TechMD retweeted
I got DeepSeek 4.1 Flash loaded on the DGX Station and running at about 80 t/s on @lmstudio's Bionic.
But local AI still needs to be much more simple to go mainstream.
TechMD retweeted
MiniMax M3.1 Flash is out! We trained a DFlash 2 drafter for it, and it's open under Apache 2.0. Generation gets up to 3.5× faster, and it's lossless.
MiniMax M3.1 Flash is out, and we've open-sourced a DFlash 2 drafter for it under Apache 2.0. It makes generation up to 3.5× faster, and it's lossless. Congrats to @MiniMax_AI on the release!
huggingface.co/incoai/MiniMa…
TechMD retweeted
Both my DGX Sparks were 95% full 😬
Now they're 84%, and not one token of the model's output changed.
Each node was hoarding ~95 GB it never even reads 🧵
TechMD retweeted
DeepSeek has opened up another piece of its efficiency playbook. DeepInfer, the inference engine powering its production services, is now available under Apache 2.0. It supports DeepSeek V4.1-Flash on both Nvidia Blackwell GPUs and Huawei Ascend 950 chips.
The code includes optimized kernels, speculative decoding and scheduling that overlaps CPU work with chip execution. Developers can now inspect and adapt the machinery DeepSeek uses to serve models at scale. It still needs components such as KV-cache infrastructure and load balancing to become a complete hosting service, but the core engine is there to study, benchmark and build on.