Cruz (@ViC305) header
Cruz (@ViC305) profile picture

God-fearing husband & father. AI Engineer. Fine-tuning + day-zero EXL3 & GGUF quants + benchmarks • Local LLMs • honest tok/s • fitting big models on small GPUs

Joined June 2009
Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)Photo by Cruz (@ViC305)
Pinned Tweet
🚨 𝗜 𝗝𝗨𝗦𝗧 𝗔𝗖𝗛𝗜𝗘𝗩𝗘𝗗 𝗦𝗢𝗠𝗘𝗧𝗛𝗜𝗡𝗚 𝗜𝗡𝗖𝗥𝗘𝗗𝗜𝗕𝗟𝗘 𝗢𝗡 𝗢𝗡𝗘 𝗗𝗚𝗫 𝗦𝗣𝗔𝗥𝗞! 🧪🔥 With the support of @AWERXAI, I'm excited to release something VERY special for the local AI community! 🧠 @deepseek_ai 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸 𝗩𝟰.𝟭 𝗙𝗹𝗮𝘀𝗵 𝟭𝟬𝟭𝟳 📦 𝗦𝗔𝗚𝗘-𝗘𝗫𝗟𝟯 𝗮𝘁 𝗷𝘂𝘀𝘁 𝟭.𝟲𝟮 𝗯𝗽𝘄 𝗲𝘅𝗽𝗲𝗿𝘁𝘀 💻 𝟵𝟵.𝟲𝟭% 𝗖𝗢𝗗𝗘 𝗮𝗻𝘀𝘄𝗲𝗿-𝘁𝗼𝗸𝗲𝗻 𝗮𝗴𝗿𝗲𝗲𝗺𝗲𝗻𝘁! ⚡ 𝟯𝟱.𝟱 𝘁𝗼𝗸/𝘀 𝗱𝗲𝗰𝗼𝗱𝗲 𝗼𝗻 𝘃𝗟𝗟𝗠! 🚀 𝟭,𝟭𝟴𝟵 𝘁𝗼𝗸/𝘀 𝗠𝗔𝗫 𝗣𝗥𝗘𝗙𝗜𝗟𝗟! 🧠 𝟮𝟮𝟵𝗞 𝗖𝗧𝗫 (𝘀𝘁𝗶𝗹𝗹 𝘄𝗼𝗿𝗸𝗶𝗻𝗴 𝘁𝗼 𝗽𝘂𝘀𝗵 𝗶𝘁 𝗳𝘂𝗿𝘁𝗵𝗲𝗿!) 𝗔𝗟𝗟 𝗢𝗡 𝗢𝗡𝗘. 𝗗𝗚𝗫. 𝗦𝗣𝗔𝗥𝗞. 🤯 And YES, the quantized weights are AVAILABLE NOW! 💚 ━━━━━━━━━━━━━━━━━━━━ 🧪 𝗧𝗛𝗘 𝗟𝗔𝗦𝗧 𝟳𝟮 𝗛𝗢𝗨𝗥𝗦 𝗛𝗔𝗩𝗘 𝗕𝗘𝗘𝗡 𝗜𝗡𝗦𝗔𝗡𝗘! Just a few days ago, I published my article about @nvidiaai's FP4-Local format: The Good, The Bad & The Ugly. One thing really stuck with me... NVIDIA demonstrated what was possible when aggressive quantization and intelligent optimization come together. But there's a problem. 𝗥𝗘𝗣𝗥𝗢𝗗𝗨𝗖𝗜𝗡𝗚 𝗠𝗔𝗦𝗦𝗜𝗩𝗘 𝗤𝗨𝗔𝗡𝗧𝗜𝗭𝗔𝗧𝗜𝗢𝗡 𝗧𝗥𝗔𝗜𝗡𝗜𝗡𝗚 𝗣𝗜𝗣𝗘𝗟𝗜𝗡𝗘𝗦 𝗜𝗦 𝗢𝗨𝗧 𝗢𝗙 𝗥𝗘𝗔𝗖𝗛 𝗙𝗢𝗥 𝗠𝗢𝗦𝗧 𝗟𝗢𝗖𝗔𝗟 𝗔𝗜 𝗘𝗡𝗧𝗛𝗨𝗦𝗜𝗔𝗦𝗧𝗦. That inspired me. I've been developing my own SAGE-EXL3 process, and I wanted to see just how far I could push the latest DeepSeek Flash release while keeping the results accessible to the community. So I spent the last 72 hours pushing the limits. And the results genuinely surprised me! 🔥 ━━━━━━━━━━━━━━━━━━━━ 🏆 𝗧𝗛𝗘 𝗥𝗘𝗦𝗨𝗟𝗧𝗦 𝗔𝗥𝗘 𝗖𝗥𝗔𝗭𝗬! I measured my 1.62 bpw expert quant against DeepSeek's native FP4 release checkpoint using held-out sequences and full-vocabulary token predictions. And when I saw the CODE results, I could barely believe it! 🧪 𝗙𝗜𝗗𝗘𝗟𝗜𝗧𝗬 𝗩𝗦 𝗗𝗘𝗘𝗣𝗦𝗘𝗘𝗞'𝗦 𝗡𝗔𝗧𝗜𝗩𝗘 𝗙𝗣𝟴/𝗙𝗣𝟰 𝗥𝗘𝗟𝗘𝗔𝗦𝗘 💻 𝗖𝗢𝗗𝗘 — 𝗛𝘂𝗺𝗮𝗻𝗘𝘃𝗮𝗹 ✅ 99.08% Top-1 Agreement ✅ 𝟵𝟵.𝟲𝟭% 𝗔𝗡𝗦𝗪𝗘𝗥-𝗢𝗡𝗟𝗬 𝗔𝗚𝗥𝗘𝗘𝗠𝗘𝗡𝗧! ✅ KL Divergence: 0.0186 🧠 𝗥𝗘𝗔𝗦𝗢𝗡𝗜𝗡𝗚 — 𝗠𝗔𝗧𝗛-𝟱𝟬𝟬 ✅ 96.04% Top-1 Agreement ✅ 97.98% Answer-Only Agreement ✅ KL Divergence: 0.0349 🔢 𝗠𝗔𝗧𝗛 — 𝗚𝗦𝗠𝟴𝗞 ✅ 95.56% Top-1 Agreement ✅ 94.19% Answer-Only Agreement ✅ KL Divergence: 0.0477 📊 𝗢𝗩𝗘𝗥𝗔𝗟𝗟 𝗛𝗘𝗟𝗗-𝗢𝗨𝗧 𝗙𝗜𝗗𝗘𝗟𝗜𝗧𝗬 🏆 94.52% Top-1 Agreement 📉 Mean KL: 0.0982 🎯 Median KL: 0.00019 𝗡𝗘𝗔𝗥-𝗟𝗢𝗦𝗦𝗟𝗘𝗦𝗦 𝗖𝗢𝗗𝗘 𝗧𝗢𝗞𝗘𝗡 𝗙𝗜𝗗𝗘𝗟𝗜𝗧𝗬 𝗔𝗧 𝗝𝗨𝗦𝗧 𝟭.𝟲𝟮 𝗕𝗣𝗪 𝗘𝗫𝗣𝗘𝗥𝗧𝗦! 🤯 These are direct teacher-forced fidelity measurements against native FP8, not task-level pass rates. But seeing this level of agreement at such aggressive compression is EXACTLY what I was working toward! ━━━━━━━━━━━━━━━━━━━━ ⚡ 𝗔𝗡𝗗 𝗬𝗘𝗦, 𝗜𝗧 𝗔𝗖𝗧𝗨𝗔𝗟𝗟𝗬 𝗥𝗨𝗡𝗦 𝗢𝗡 𝗢𝗡𝗘 𝗦𝗣𝗔𝗥𝗞! Not two Sparks. Not four Sparks. Not an expensive rented GPU cluster. 𝗢𝗡𝗘 𝟭𝟮𝟴𝗚𝗕 𝗗𝗚𝗫 𝗦𝗣𝗔𝗥𝗞. My latest vLLM measurements: ⚡ Decode: 35.5 tok/s 🚀 Maximum measured prefill: 1,189 tok/s For a model this large, compressed this aggressively, and maintaining this level of fidelity... 𝗧𝗛𝗔𝗧'𝗦 𝗔 𝗕𝗜𝗚 𝗗𝗘𝗔𝗟 𝗙𝗢𝗥 𝗟𝗢𝗖𝗔𝗟 𝗔𝗜! 💚 And remember, 1.62 bpw refers to the routed experts, not every parameter in the full model. ━━━━━━━━━━━━━━━━━━━━ 💚 𝗪𝗛𝗬 𝗧𝗛𝗜𝗦 𝗠𝗔𝗧𝗧𝗘𝗥𝗦. I have a simple goal. 𝗠𝗔𝗞𝗘 𝗣𝗢𝗪𝗘𝗥𝗙𝗨𝗟 𝗔𝗜 𝗔𝗖𝗖𝗘𝗦𝗦𝗜𝗕𝗟𝗘 𝗧𝗢 𝗣𝗘𝗢𝗣𝗟𝗘 𝗪𝗛𝗢 𝗗𝗢𝗡'𝗧 𝗛𝗔𝗩𝗘 𝗔𝗖𝗖𝗘𝗦𝗦 𝗧𝗢 𝗗𝗔𝗧𝗔𝗖𝗘𝗡𝗧𝗘𝗥𝗦. I don't want the future of local AI to be determined by who can rent the most GPUs or afford the largest training runs. I want developers, independent researchers and enthusiasts to have the freedom to experiment, quantize, optimize and run powerful models on hardware they actually OWN. NVIDIA's FP4-Local research showed me just how much potential remains in aggressive quantization. My goal with SAGE-EXL3 is to keep pushing that frontier while making advanced quantization more practical for the wider community. Today, I'm sharing the quantized checkpoint and the one-Spark serving recipe. 𝗧𝗛𝗜𝗦 𝗜𝗦 𝗝𝗨𝗦𝗧 𝗧𝗛𝗘 𝗕𝗘𝗚𝗜𝗡𝗡𝗜𝗡𝗚. ━━━━━━━━━━━━━━━━━━━━ 🧪 𝗡𝗢𝗪 𝗙𝗢𝗥 𝗧𝗛𝗘 𝗥𝗘𝗔𝗟 𝗧𝗘𝗦𝗧! The token-level fidelity results are incredible, but I'm not stopping there. I'm currently running a broader evaluation campaign to measure how closely this quant preserves the native FP8 model's actual task performance. 💻 HumanEval+ / MBPP+ 🧠 MMLU-Pro STEM / MATH-500 🔬 SciCode / LiveCodeBench 🔧 BFCL / Multi-step tool chains 🌍 MGSM 𝗧𝗛𝗘 𝗚𝗢𝗔𝗟 𝗜𝗦 𝗧𝗢 𝗣𝗥𝗢𝗩𝗘 𝗧𝗛𝗔𝗧 𝗔𝗚𝗚𝗥𝗘𝗦𝗦𝗜𝗩𝗘 𝗖𝗢𝗠𝗣𝗥𝗘𝗦𝗦𝗜𝗢𝗡 𝗗𝗢𝗘𝗦𝗡'𝗧 𝗛𝗔𝗩𝗘 𝗧𝗢 𝗠𝗘𝗔𝗡 𝗔𝗚𝗚𝗥𝗘𝗦𝗦𝗜𝗩𝗘 𝗤𝗨𝗔𝗟𝗜𝗧𝗬 𝗟𝗢𝗦𝗦! Those task-level results are coming next! 👀 ━━━━━━━━━━━━━━━━━━━━ 👀 𝗔𝗡𝗗 𝗢𝗡𝗘 𝗟𝗜𝗧𝗧𝗟𝗘 𝗧𝗘𝗔𝗦𝗘𝗥... @mr_r0b0t and I have also been working on an GB10 specialized inference engine called 𝗥𝗘𝗫𝗟𝟯. Early testing has been VERY exciting, including 60 tok/s with bursts reaching 74tok/s! We're putting the finishing touches on it and making sure everything is solid before releasing it. More on that VERY soon. 😏 ━━━━━━━━━━━━━━━━━━━━ 🙏 𝗔 𝗛𝗨𝗚𝗘 𝗧𝗛𝗔𝗡𝗞 𝗬𝗢𝗨! To @AWERXAI for supporting my work and helping make this contribution to the community possible. To @nvidiaai for the FP4-Local research that inspired me to push this experiment even further. To @justinn_builds for his initial vllm TP4 recipe that got me started in the right direction for loading this model up. To @mr_r0b0t for collaborating with me on what's coming next! And to @deepseek_ai for continuing to release powerful open-weight models that make community experiments like this possible. 𝗧𝗛𝗘 𝗕𝗘𝗦𝗧 𝗣𝗔𝗥𝗧 𝗢𝗙 𝗢𝗣𝗘𝗡 𝗔𝗜 𝗜𝗦 𝗪𝗛𝗔𝗧 𝗧𝗛𝗘 𝗖𝗢𝗠𝗠𝗨𝗡𝗜𝗧𝗬 𝗖𝗔𝗡 𝗕𝗨𝗜𝗟𝗗 𝗢𝗡 𝗧𝗢𝗣 𝗢𝗙 𝗜𝗧. 💚 🔗 𝗗𝗢𝗪𝗡𝗟𝗢𝗔𝗗 𝗠𝗬 𝗦𝗔𝗚𝗘-𝗘𝗫𝗟𝟯 huggingface.co/vcruz305/DSV4… 🛠️ 𝗢𝗡𝗘-𝗦𝗣𝗔𝗥𝗞 𝗥𝗘𝗖𝗜𝗣𝗘 github.com/vcruz305/DSV4.1-F… 𝗢𝗡𝗘 𝗦𝗣𝗔𝗥𝗞. 𝟭.𝟲𝟮 𝗕𝗣𝗪 𝗘𝗫𝗣𝗘𝗥𝗧𝗦. 𝟵𝟵.𝟲𝟭% 𝗖𝗢𝗗𝗘 𝗔𝗡𝗦𝗪𝗘𝗥 𝗔𝗚𝗥𝗘𝗘𝗠𝗘𝗡𝗧. 𝗟𝗘𝗧'𝗦 𝗞𝗘𝗘𝗣 𝗣𝗨𝗦𝗛𝗜𝗡𝗚 𝗟𝗢𝗖𝗔𝗟 𝗔𝗜 𝗙𝗢𝗥𝗪𝗔𝗥𝗗! 🚀🔥
75
57
32
384
48,358
It’s extremely important that the open sourced community is made aware of all quantization formats that can provide the most value for the least amount of GPU/VRAM and that’s where EXL3 by @turboderp_ comes in to shine. I’ve coined my own EXL3 quantization process called SAGE(Sentinel Assisted GPU-aware EXL3) that has grown over the last few months into a contender that can score higher than even its 8bit GGUF counter parts with Deepseek v4.1 Flash 1017 being the model in the spotlight, while running on one DGX Spark. I’ve been able to achieve a 94% top agreement at 1.55Bpw effectively breaking into the One DGX Spark Pareto Frontier. With that being said, here’s an article of proof that I, a one man team with the resources of a company I work for that’s currently in stealth, put together to demonstrate the strengths and weaknesses of my quantization process on how a sub 2bit quant can exceed the intelligence and quality of 2-8bit quantizations in GGUF. You’ll also find a HuggingFace dataset of proofs and results! Thank you to @UnslothAI & @mhelcig for bringing up this unsolicited challenge by pushing the capabilities of GGUF forward and @NVIDIAAI @NVIDIARTXSpark for bringing frontier model access to our desk by ways of the RTX and DGX Spark platforms! Here’s to fitting more of what counts in the GB10 🫡🙏 🏆 HMMT: 70.27% above all 8. 🧮 AIME: 95.42% above 5/8. 📋 IFEval: 92.05% above 1/8. Article: nitter.gq/vic305/status/21173776…
🤖 Made with AI
We benchmarked DeepSeek-V4.1-Flash-1017 GGUFs on Terminal Bench, DeepSWE & more on 512K context. Unsloth ranks highest overall. 1-bit Unsloth GGUF scores 68.1 on DeepSWE, 72.2 on Terminal-Bench, 52.7 on AutomationBench. >10% ahead of other quants. GGUF: huggingface.co/unsloth/DeepS…
1
10
40
1,969
🔥 1.55 BPW DeepSeek SAGE-EXL3 vs @UnslothAI GGUF & GSQ-RCO! 117.2GB backbone. 🏆 HMMT: 70.27%—above all 8 (retries ongoing). 🧮 AIME: 95.42%—above 5/8. 📋 IFEval: 92.05%—above 1/8. Full six-benchmark showdown + one-Spark 262K goal! 🚀👇
3
4
1
21
1,236
It’s an honor and a pleasure to be mentioned by @MiniMax_AI for my hard work on my SAGE-EXL3 quantization of their latest MiniMax M3.1 Flash! I really worked hard to get this model at max fidelity into one spark ! If anyone’s interested in testing out my quant here’s the HF 😁 96% top / 0.02 KLD huggingface.co/vcruz305/Mini… Recipe: github.com/vcruz305/MiniMax-…
MiniMax M3.1 Flash, on more of your hardware. Quantizing a 428B model opens up more ways to run it locally. Lower-bit weights reduce the memory needed to load the model and the weight data moved during decoding. Making those savings useful takes careful calibration, efficient low-bit kernels, and hardware-aware placement of experts across GPU and system RAM. Thank you to the builders doing that work: • @UnslothAI @danielhanchen — 2-bit Dynamic GGUF weights at 143.9 GB, with calibration and fidelity testing across coding, tool use and other workloads. • @ViC305 — SAGE-EXL3, token-level fidelity measurements and a single-DGX-Spark recipe. • @PeasantSmith — Project Maya quants and expert offloading for local coding and tool use. • @MiaAI_lab — EXL3-EAQ quantization work. • @jundotkim + mlx-community — oQ quants and DSpark-powered decoding through oMLX on Apple Silicon. • @AIatAMD — Quark MXFP4 adapted for AMD Instinct. • @atomic_chat_hq — GGUFs for local desktop workflows. • @superalesha — custom inference work combining quantized weights, expert caching and system RAM on 4× RTX 3090. Every shared quant, kernel, measurement and recipe gives the next builder a better starting point. Thank you.
7
3
45
2,231
🚨 𝗩𝗘𝗥𝗜𝗙𝗜𝗘𝗗 𝗛𝗢𝗔𝗫: 𝗧𝗛𝗘 𝟳.𝟳𝟮𝗧𝗕 “𝗔𝗡𝗧𝗛𝗥𝗢𝗣𝗜𝗖 𝗡𝗨𝗠𝗕𝗔𝗧” 𝗪𝗘𝗜𝗚𝗛𝗧 𝗟𝗘𝗔𝗞 𝗜𝗦 𝗙𝗔𝗞𝗘! ❌ Someone asked whether anybody had actually inspected the weights. 𝗬𝗘𝗦. 𝗜 𝗗𝗜𝗗. And we didn’t need to download 7.72 TERABYTES or rent a single GPU to expose it! 🤯 As someone who spends an enormous amount of time quantizing frontier models, one of the most important skills I’ve developed is knowing how to inspect and validate raw model weights BEFORE spending resources on them. So when this supposed ~14 TRILLION parameter Anthropic checkpoint surfaced, naturally I got curious. I had my agents investigate its structure, tensor distributions and provenance. And the strongest evidence is now independently reproducible! ━━━━━━━━━━━━━━━━━━━━ 🔬 𝗣𝗥𝗢𝗢𝗙 #𝟭 — 𝗧𝗛𝗘 𝗪𝗘𝗜𝗚𝗛𝗧𝗦 𝗖𝗔𝗡 𝗕𝗘 𝗚𝗘𝗡𝗘𝗥𝗔𝗧𝗘𝗗 𝗙𝗥𝗢𝗠 𝗧𝗛𝗘𝗜𝗥 𝗢𝗪𝗡 𝗡𝗔𝗠𝗘𝗦! An independent Hugging Face auditor, FloxUnder, reproduced: ✅ 43 COMPLETE tensors ✅ 730,368 individual BF16 values ✅ 100% BYTE-FOR-BYTE MATCH! The recipe? 1️⃣ SHA256 hash the tensor name. 2️⃣ Use the hash to seed NumPy PCG64. 3️⃣ Generate deterministic random values. 4️⃣ Apply simple mathematical transformations. 5️⃣ Convert to BF16. 𝗧𝗛𝗘 𝗥𝗘𝗦𝗨𝗟𝗧? 𝗘𝗫𝗔𝗖𝗧𝗟𝗬 𝗧𝗛𝗘 𝗦𝗔𝗠𝗘 𝗪𝗘𝗜𝗚𝗛𝗧 𝗕𝗬𝗧𝗘𝗦. I independently reran the published reproduction on a complete 12,288-value normalization tensor and a 24,576-value convolution tensor. 𝗕𝗢𝗧𝗛 𝗠𝗔𝗧𝗖𝗛𝗘𝗗. 𝗘𝗫𝗔𝗖𝗧𝗟𝗬. And the auditor even predicted previously unread tensors before downloading their bytes! A genuine trained checkpoint’s learned parameters should not be reproducible solely from their tensor names and a simple PRNG recipe. That’s an enormous red flag against the claimed provenance. ━━━━━━━━━━━━━━━━━━━━ 🧠 𝗣𝗥𝗢𝗢𝗙 #𝟮 — 𝗧𝗛𝗘 𝗡𝗢𝗥𝗠𝗔𝗟𝗜𝗭𝗔𝗧𝗜𝗢𝗡 𝗪𝗘𝗜𝗚𝗛𝗧𝗦 𝗗𝗢𝗡’𝗧 𝗟𝗢𝗢𝗞 𝗧𝗥𝗔𝗜𝗡𝗘𝗗. Another independent analysis compared Numbat’s normalization tensors against real trained Qwen and DeepSeek checkpoints. Across sampled layers, Numbat’s weights exhibit strikingly uniform synthetic-looking distributions instead of the learned structure expected from extensive training. Combined with the EXACT PRNG reproduction, the evidence strongly points to generated tensors being presented as trained model weights. ━━━━━━━━━━━━━━━━━━━━ 📦 𝗣𝗥𝗢𝗢𝗙 #𝟯 — 𝟳.𝟳𝟮𝗧𝗕 𝗢𝗙 𝗙𝗜𝗟𝗘𝗦 𝗗𝗢𝗘𝗦𝗡’𝗧 𝗠𝗘𝗔𝗡 𝟳.𝟳𝟮𝗧𝗕 𝗢𝗙 𝗜𝗡𝗧𝗘𝗟𝗟𝗜𝗚𝗘𝗡𝗖𝗘. 772 shards. Plausible tensor names. Matching metadata. A sophisticated-looking model architecture. None of those establishes that a model was actually trained—or that Anthropic created it. Even the genuine appearance of the Numbat codename in earlier Claude Code source material doesn’t authenticate this checkpoint. 𝗠𝗘𝗧𝗔𝗗𝗔𝗧𝗔 𝗖𝗔𝗡 𝗕𝗘 𝗙𝗔𝗕𝗥𝗜𝗖𝗔𝗧𝗘𝗗. 𝗦𝗛𝗔𝗥𝗗𝗦 𝗖𝗔𝗡 𝗕𝗘 𝗚𝗘𝗡𝗘𝗥𝗔𝗧𝗘𝗗. 𝗟𝗘𝗔𝗥𝗡𝗘𝗗 𝗪𝗘𝗜𝗚𝗛𝗧𝗦 𝗡𝗘𝗘𝗗 𝗥𝗘𝗔𝗟 𝗘𝗩𝗜𝗗𝗘𝗡𝗖𝗘. ━━━━━━━━━━━━━━━━━━━━ 💡 𝗧𝗛𝗘 𝗧𝗔𝗞𝗘𝗔𝗪𝗔𝗬 𝗙𝗢𝗥 𝗧𝗛𝗘 𝗔𝗜 𝗖𝗢𝗠𝗠𝗨𝗡𝗜𝗧𝗬: Before downloading a massive mystery checkpoint, inspect the index, read small tensor ranges, examine their statistics and check whether supposedly learned values can be regenerated. Don’t blindly execute untrusted model code just to investigate a supposed leak! The strongest verification here required only a tiny fraction of the claimed download and NO GPU. We don’t know who created this repository or why. And this investigation doesn’t suggest Anthropic was responsible for it. Huge thanks to @Blackfrost_AI for putting this on my radar! 🔗 𝗘𝗫𝗔𝗖𝗧-𝗕𝗬𝗧𝗘 𝗣𝗥𝗢𝗢𝗙 + 𝗥𝗘𝗣𝗥𝗢𝗗𝗨𝗖𝗧𝗜𝗢𝗡 𝗖𝗢𝗗𝗘 huggingface.co/burner22/numb… 🔗 𝗜𝗡𝗗𝗘𝗣𝗘𝗡𝗗𝗘𝗡𝗧 𝗧𝗘𝗡𝗦𝗢𝗥 𝗔𝗡𝗔𝗟𝗬𝗦𝗜𝗦 sysnote.net/news/the-alleged… 🔗 𝗥𝗘𝗣𝗢 𝗨𝗡𝗗𝗘𝗥 𝗜𝗡𝗩𝗘𝗦𝗧𝗜𝗚𝗔𝗧𝗜𝗢𝗡 huggingface.co/burner22/numb… 𝗔 𝗣𝗔𝗥𝗔𝗠𝗘𝗧𝗘𝗥 𝗖𝗢𝗨𝗡𝗧 𝗜𝗦 𝗡𝗢𝗧 𝗣𝗥𝗢𝗢𝗙 𝗢𝗙 𝗧𝗥𝗔𝗜𝗡𝗜𝗡𝗚.
THIS COULD BE ONE OF THE CRAZIEST AI LEAKS EVER 💀 AN ALLEGED MYTHOS-CLASS MODEL WITH 14 TRILLION PARAMETERS JUST APPEARED ON HUGGING FACE. BUT THERE'S ONE PROBLEM... NOBODY HAS VERIFIED THE WEIGHTS. HERE'S WHAT THE REPO CLAIMS: - 14 TRILLION PARAMETERS - 1,224 MoE EXPERTS - 7.72 TB OF WEIGHTS - 772 WEIGHT SHARDS - HYBRID GDN + MLA ARCHITECTURE - MODEL CONFIGS, ARCHITECTURE CODE AND TRAINING METADATA IF THESE CLAIMS ARE REAL, THIS COULD BE MASSIVE. BUT A REPO CLAIMING TO CONTAIN INTERNAL WEIGHTS DOESN'T PROVE THEY'RE AUTHENTIC. ANYONE LOOKED INTO THE ACTUAL FILES YET? LEGIT LEAK OR ELABORATE FAKE? 👀 huggingface.co/burner22/numb…
Readers added context they thought people might want to know
The Hugging Face repo burner22/numbat makes unverified claims of an Anthropic "numbat" Mythos-class leak with ~14T params/7.72TB weights, but contains fabricated weights (inconsistent 6.8T metadata; distributions match synthetic data, not trained models like DeepSeek). huggingface.co/burner22/numbat x.com/EricParker/sta… en.wikipedia.org/wiki/Claude_My…
5
6
4
38
6,031
🚨 𝗛𝗘𝗥𝗠𝗘𝗦 𝗠𝗢𝗕𝗜𝗟𝗘 𝗕𝗘𝗧𝗔 — 𝗦𝗘𝗟𝗙-𝗛𝗢𝗦𝗧𝗘𝗗 𝗦𝗘𝗧𝗨𝗣 𝗚𝗨𝗜𝗗𝗘! 📱🔥 I've spent over a DECADE in systems engineering, and somehow getting the new Hermes Mobile Beta connected to my own self-hosted @NousResearch Hermes Agent still gave me a little trouble! 😂 So here's a quick guide to save everyone else the headache! ━━━━━━━━━━━━━━━━━━━━ 🖥️ 𝗦𝗧𝗘𝗣 𝟭 — 𝗦𝗧𝗔𝗥𝗧 𝗧𝗛𝗘 𝗗𝗔𝗦𝗛𝗕𝗢𝗔𝗥𝗗 On the machine running Hermes: hermes dashboard --host 0.0.0.0 --port 9119 --no-open On first interactive setup, Hermes can prompt you to create a username and password. SAVE THOSE! 🔒 𝗣𝗥𝗘𝗙𝗘𝗥 𝗧𝗔𝗜𝗟𝗦𝗖𝗔𝗟𝗘? (Recommended) Bind directly to your machine's Tailscale IP instead: hermes dashboard --host 100.x.x.x --port 9119 --no-open ━━━━━━━━━━━━━━━━━━━━ 📱 𝗦𝗧𝗘𝗣 𝟮 — 𝗚𝗥𝗔𝗕 𝗧𝗛𝗘 𝗕𝗘𝗧𝗔 hermes-agent.nousresearch.co… ⚙️ 𝗦𝗧𝗘𝗣 𝟯 — 𝗦𝗘𝗟𝗘𝗖𝗧 𝗦𝗘𝗟𝗙-𝗛𝗢𝗦𝗧𝗘𝗗 NOT Nous Portal! You're connecting to your OWN Hermes instance. 🌐 𝗦𝗧𝗘𝗣 𝟰 — 𝗘𝗡𝗧𝗘𝗥 𝗬𝗢𝗨𝗥 𝗚𝗔𝗧𝗘𝗪𝗔𝗬 𝗨𝗥𝗟 🏠 Home Wi-Fi: http://192.168.x.x:9119 🔐 Tailscale: http://100.x.x.x:9119 Replace the example IP with your Hermes machine's actual address. 🔑 𝗦𝗧𝗘𝗣 𝟱 — 𝗘𝗡𝗧𝗘𝗥 𝗬𝗢𝗨𝗥 𝗖𝗥𝗘𝗗𝗘𝗡𝗧𝗜𝗔𝗟𝗦 Use the dashboard username and password you configured earlier. ✅ 𝗦𝗧𝗘𝗣 𝟲 — 𝗧𝗔𝗣 𝗖𝗢𝗡𝗡𝗘𝗖𝗧! That's it. Your self-hosted Hermes Agent, accessible from your phone! 🙌 ━━━━━━━━━━━━━━━━━━━━ 💡 𝗧𝗪𝗢 𝗜𝗠𝗣𝗢𝗥𝗧𝗔𝗡𝗧 𝗧𝗜𝗣𝗦: • Your phone must be on the same reachable network or connected through Tailscale. • Never expose port 9119 directly to the public internet with basic password authentication. I recommend Tailscale! Hopefully this saves someone else the troubleshooting session I went through! 😂 Enjoy Hermes Mobile! 📱💚 This thing packs a punch and even connects to your Hermes desktop sessions, something I’ve been in DESPERATE need of!! 🔥
30
22
9
317
27,362
🚨 𝗠𝗬 𝗗𝗘𝗘𝗣𝗦𝗘𝗘𝗞 𝗘𝗫𝗟𝟯: ~𝟭𝟮𝟯 𝗚𝗕 | 𝟵𝟱% 𝗠𝗠𝗟𝗨-𝗣𝗿𝗼 𝗦𝗧𝗘𝗠! VS @UnslothAI 𝗤𝟮 𝗚𝗚𝗨𝗙: ~𝟮𝟬𝟬 𝗚𝗕 | 𝟴𝟴.𝟮𝟴% 𝗠𝗠𝗟𝗨-𝗣𝗿𝗼! 👀🔥 𝗦𝗔𝗠𝗘 𝗗𝗘𝗘𝗣𝗦𝗘𝗘𝗞 𝗠𝗢𝗗𝗘𝗟. 𝗗𝗜𝗙𝗙𝗘𝗥𝗘𝗡𝗧 𝗙𝗢𝗥𝗠𝗔𝗧𝗦. 𝗕𝗜𝗚 𝗤𝗨𝗘𝗦𝗧𝗜𝗢𝗡… 𝗪𝗛𝗬 𝗜𝗦𝗡’𝗧 𝗘𝗫𝗟𝟯 𝗢𝗡 𝗧𝗛𝗘 𝗖𝗛𝗔𝗥𝗧?! 🧪 Now, those are different MMLU-Pro subsets and sampling configurations, NOT a matched accuracy comparison. But that’s exactly why I want to put them head-to-head! My DeepSeek V4.1 Flash 1017 SAGE-EXL3 is already running on the DGX Spark sitting at home. And the results deserve a place in this conversation! ━━━━━━━━━━━━━━━━━━━━ 🏆 𝗠𝗬 𝟭.𝟲𝟮 𝗕𝗣𝗪 𝗦𝗔𝗚𝗘-𝗘𝗫𝗟𝟯 🧠 MMLU-Pro STEM: 95/100 🔢 MATH-500: 40/40 💻 HumanEval+: 39/40 ⚙️ MBPP+: 53/60 🏆 LiveCodeBench: 15/17 𝗢𝗩𝗘𝗥𝗔𝗟𝗟: 𝟵𝟰.𝟭% 𝗔𝗖𝗥𝗢𝗦𝗦 𝗠𝗬 𝟵-𝗦𝗨𝗜𝗧𝗘, 𝟯𝟱𝟯-𝗜𝗧𝗘𝗠 𝗧𝗘𝗦𝗧 𝗦𝗘𝗧! That’s at just 1.62 bpw for the routed experts, not a blanket bitrate for every tensor. ━━━━━━━━━━━━━━━━━━━━ 📦 𝗡𝗢𝗪 𝗟𝗘𝗧’𝗦 𝗧𝗔𝗟𝗞 𝗦𝗜𝗭𝗘. Excluding Engram tables from EVERY configuration: 🟢 My SAGE-EXL3: ~123 GB 🔵 Unsloth UD-IQ2_XXS: 165.3 GB 🟣 Unsloth UD-Q2_K_XL: 199.6 GB That’s approximately 26–38% FEWER non-Engram weight bytes than those two GGUF variants! 🤯 These are weight-size comparisons, NOT peak runtime memory measurements or complete download sizes. And Unsloth’s UD-IQ1_S comes in at 122.4 GB on the same basis, making it another VERY interesting equal-size contender. 𝗧𝗛𝗔𝗧’𝗦 𝗧𝗛𝗘 𝗖𝗢𝗠𝗣𝗔𝗥𝗜𝗦𝗢𝗡 𝗜 𝗪𝗔𝗡𝗧 𝗧𝗢 𝗦𝗘𝗘! ━━━━━━━━━━━━━━━━━━━━ ⚡ 𝗔𝗡𝗗 𝗧𝗛𝗜𝗦 𝗜𝗦𝗡’𝗧 𝗝𝗨𝗦𝗧 𝗔 𝗤𝗨𝗔𝗡𝗧 𝗦𝗜𝗧𝗧𝗜𝗡𝗚 𝗢𝗡 𝗛𝗨𝗚𝗚𝗜𝗡𝗚 𝗙𝗔𝗖𝗘! My public vLLM serving configuration has separately measured: 🚀 35.5 tok/s decode 🖥️ ONE 128GB DGX Spark 🧠 Over 500K context tested successfully! The nine-suite quality measurements above used REXL3, while the speed result is from vLLM. Different engines, separately reported. ━━━━━━━━━━━━━━━━━━━━ 🔬 𝗡𝗢𝗪 𝗙𝗢𝗥 𝗧𝗛𝗘 𝗥𝗘𝗔𝗟 𝗛𝗘𝗔𝗗-𝗧𝗢-𝗛𝗘𝗔𝗗. My current results use matched subsets at temperature 0. Unsloth’s published MMLU-Pro result uses a larger 1,203-item subset with different sampling and grading conditions. So I’m targeting aligned evaluations across: 🧠 MMLU-Pro 🔢 AIME25 🔬 GPQA Diamond Same questions. Same sampling. Same reasoning budget. Same grading. 𝗟𝗘𝗧’𝗦 𝗦𝗘𝗘 𝗛𝗢𝗪 𝗠𝗨𝗖𝗛 𝗜𝗡𝗧𝗘𝗟𝗟𝗜𝗚𝗘𝗡𝗖𝗘 𝗘𝗔𝗖𝗛 𝗙𝗢𝗥𝗠𝗔𝗧 𝗗𝗘𝗟𝗜𝗩𝗘𝗥𝗦 𝗣𝗘𝗥 𝗚𝗕. GGUF has earned its place in local AI. EXL3 deserves to be in that conversation too. 𝗚𝗚𝗨𝗙 𝘃𝘀 𝗘𝗫𝗟𝟯. 𝗡𝗢𝗧 𝗝𝗨𝗦𝗧 𝗚𝗚𝗨𝗙 𝘃𝘀 𝗚𝗚𝗨𝗙. Bring on the benchmarks! ⚡🔥 🔗 𝗠𝗬 𝗦𝗔𝗚𝗘-𝗘𝗫𝗟𝟯 𝗪𝗘𝗜𝗚𝗛𝗧𝗦 huggingface.co/vcruz305/DSV4… 🛠️ 𝗢𝗡𝗘-𝗦𝗣𝗔𝗥𝗞 𝗥𝗘𝗖𝗜𝗣𝗘 github.com/vcruz305/DSV4.1-F…
We’re releasing new DeepSeek-V4.1-Flash-1017 GGUFs, ranking highest across benchmarks. Dynamic V3.1 leads across nearly all benchmarks: GPQA Diamond, AIME25, LiveCodeBench v6, MMLU-Pro, Aider & more. Run on 64GB+ RAM. GGUF: huggingface.co/unsloth/DeepS… Blog: unsloth.ai/docs/models/deeps…
11
6
2
53
6,280
🚨 I just squeezed Meta’s 816B Muse into 𝗝𝗨𝗦𝗧 ~𝟭𝟮𝟬 𝗚𝗕 at 𝟭.𝟭𝟯 𝗯𝗽𝘄, with 𝟵𝟲.𝟰𝟴% 𝘁𝗼𝗽-𝟭 𝘁𝗼𝗸𝗲𝗻 𝗮𝗴𝗿𝗲𝗲𝗺𝗲𝗻𝘁 and 𝟬.𝟬𝟱𝟱𝟬𝟭 𝗺𝗲𝗮𝗻 𝗞𝗟𝗗! 🤯🔥 @AIatMeta @MetaforDevs @finkd I had to see how far SAGE-EXL3 could take Muse Spark Open! Today I’m releasing my most extreme quantization yet: 𝗠𝘂𝘀𝗲-𝗦𝗽𝗮𝗿𝗸-𝗢𝗽𝗲𝗻-𝟴𝟭𝟲𝗕-𝗔𝟰𝟮𝗕-𝗘𝗫𝗟𝟯-𝗦𝗔𝗚𝗘-𝟭.𝟭𝟯𝗯𝗽𝘄 🧠 816 BILLION total parameters / 42B active 📦 111.81 GiB of quantized weights — approximately 120 GB 🧪 1.13 bpw SAGE-EXL3 Not another 4-bit release. Not even 2-bit. 𝗪𝗘’𝗥𝗘 𝗚𝗘𝗧𝗧𝗜𝗡𝗚 𝗖𝗟𝗢𝗦𝗘 𝗧𝗢 𝗧𝗛𝗘 𝗢𝗡𝗘-𝗕𝗜𝗧 𝗟𝗜𝗡𝗘. 👀 ━━━━━━━━━━━━━━━━━━━━ 🎯 𝗘𝗫𝗧𝗥𝗘𝗠𝗘 𝗖𝗢𝗠𝗣𝗥𝗘𝗦𝗦𝗜𝗢𝗡. 𝗦𝗘𝗥𝗜𝗢𝗨𝗦 𝗙𝗜𝗗𝗘𝗟𝗜𝗧𝗬. ✅ 96.48% Top-1 Token Agreement ✅ 99.78% Top-5 Agreement 📉 0.05501 Mean KL Divergence 🎯 0.00010 Median KL Divergence Measured against the unquantized BF16 reference on held-out sequences, across the full vocabulary. These are token-level fidelity measurements, not task pass rates or a claim of literal losslessness. But preserving this much of the reference model’s predictions at such an aggressive bitrate is EXACTLY what makes this release exciting! 𝗧𝗛𝗘 𝗙𝗜𝗟𝗘 𝗦𝗜𝗭𝗘 𝗚𝗘𝗧𝗦 𝗬𝗢𝗨𝗥 𝗔𝗧𝗧𝗘𝗡𝗧𝗜𝗢𝗡. 𝗧𝗛𝗘 𝗙𝗜𝗗𝗘𝗟𝗜𝗧𝗬 𝗠𝗔𝗞𝗘𝗦 𝗜𝗧 𝗪𝗢𝗥𝗧𝗛 𝗘𝗫𝗣𝗟𝗢𝗥𝗜𝗡𝗚. 🔥 For me, this opens an exciting new chapter for EXL3 and sub-2-bit quantization. ━━━━━━━━━━━━━━━━━━━━ 🛠️ 𝗪𝗘𝗜𝗚𝗛𝗧𝗦 𝗡𝗢𝗪. 𝗧𝗪𝗢-𝗦𝗣𝗔𝗥𝗞 𝗥𝗘𝗖𝗜𝗣𝗘 𝗡𝗘𝗫𝗧. The quant is published. I’m still working on a proper serving recipe for TWO DGX Sparks. The ~120 GB figure is the checkpoint size, not the complete running memory footprint. The inference engine and KV cache need room too. Stay tuned for the validated setup, memory requirements and measured performance. I want the recipe to be something you can actually follow, not just a screenshot of a successful boot. Thank you to the Meta team for releasing Muse Spark Open and giving the community something this ambitious to build on! 💚 🔗 𝗠𝗢𝗗𝗘𝗟 & 𝗙𝗜𝗗𝗘𝗟𝗜𝗧𝗬 𝗥𝗘𝗦𝗨𝗟𝗧𝗦 huggingface.co/vcruz305/Muse… 𝗣𝘂𝘀𝗵𝗲𝗱 𝘁𝗼 𝟭.𝟭𝟯 𝗯𝗽𝘄 𝘄𝗶𝘁𝗵 𝗦𝗔𝗚𝗘-𝗘𝗫𝗟𝟯. 🧪🔥
19
4
4
91
5,957
I've put over 3 billion tokens through Gemini Argon in the two weeks since it came out. Now @GoogleDeepMind already has Gemini 4 Carbon, and it's the new #1 on the Artificial Analysis Intelligence Index at 59. That's a crazy pace. It doesn't win everything though. Claude Opus 5.5 still leads their coding agent index, 62 to 59, so "best model" really depends on what you're doing all day. 1M context, text, image, video and speech in, $4/$20 per million tokens with a 50% launch discount. 11% hallucination rate is the number I'll be watching once I'm running it. Can't wait for this one to hit @OpenRouter!!
Google’s Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5, and second only to Claude Opus 5.5 on the Artificial Analysis Coding Agent Index Gemini 4 Carbon scores 59 on Intelligence Index v5, driven by leading performance in complex reasoning, knowledge, and scientific tasks. It also has large gains in coding and knowledge work, particularly when compared to other Gemini models, but it is not leading in all these areas. As with launch of Gemini 4 Argon, Gemini 4 Carbon is being rolled out to selected users and is not yet publicly available. Google is offering a 50% launch discount, with no confirmed end date. All our charts and figures use standard pricing for both Gemini 4 Carbon and Argon. Key results for Gemini 4 Carbon with high reasoning: ➤ Gemini 4 Carbon leads the Intelligence Index v5, with the #1 score in 4 of the 9 Index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, AutomationBench-AA. Carbon makes substantial gains elsewhere but is not leading in coding and knowledge work ➤ Behind only Claude Opus 5.5 on coding agents: With Antigravity CLI, Gemini 4 Carbon (high) scores 59 only 3 points behind Claude Code with Claude Opus 5.5 (max, 62) on the Artificial Analysis Coding Agent Index. Carbon ranks second on both Terminal-Bench 4.0 (64%) and FrontierCode v1.1 (51%), behind Claude Opus 5.5 (max) at 56%. Google has historically underperformed in this area and so this improvement stands out ➤ Improved agentic knowledge work, but not yet leading: On AA-Briefcase, Gemini 4 Carbon (high) scores 1609 Elo, a 126-point improvement over Gemini 4 Argon (high, 1483), but still behind models including Grok 4.7 (xhigh, 1638), Muse Wonder (max, 1659) and Kimi K3.1 Pro (1678). The largest gain comes from Presentation, up 293 Elo to 1598, alongside a 120-point improvement in Analytical Quality to 1671 ➤ Lower Cost per Task than Claude Opus 5.5 for comparable overall intelligence: Gemini 4 Carbon costs $7.76 per Intelligence Index task, ~23% less than Claude Opus 5.5 (max, $10.04) but ~1.2x GPT-6 Astra (max, $6.25). It uses 123k output tokens per task, 29% fewer than Claude Opus 5.5 (max, 173k). At the same $4/$20 standard pricing, its Cost per Task is in line with Gemini 4 Argon ($7.68), while scoring 9 points higher on the Intelligence Index ➤ Leading AA-Omniscience v1.1 score, with low hallucination: Gemini 4 Carbon scores 54 on our recently updated AA-Omniscience v1.1, 6 points ahead of Claude Opus 5.5 (max, 48) and 8 points ahead of Gemini 4 Argon (46). Accuracy (% correct) rises 10 points over Argon to 58%, while its hallucination rate rises from 4% to 11%. This is still the second lowest of any model, behind only Argon, compared with 50% for both Claude Opus 5.5 (max) and GPT-6 Astra (max) Key model details: ➤ Context Window: 1M tokens ➤ Multimodality: Text, image, video, and speech input, with text output ➤ Pricing: $4/$20 per 1M input/output tokens, the same as Gemini 4 Argon’s standard pricing, with a 50% discount. Cached input tokens receive a 95% discount ($0.20 per 1M) Congratulations to the @GoogleDeepMind team!
5
2
15
3,827
Got pre-release access to Nemotron 3.5 Transcribe from @NVIDIAAI and the speaker identification is the part I love the most!! 30B MoE with only ~4B active, 40 languages, fully open weights, data and recipes. It just took #1 on the Open ASR leaderboard at 5.33 avg WER, ahead of Qwen3-ASR 1.7B and IBM Granite Speech. What I keep thinking about is pairing it with an app like Meetily for local meeting notes. On a GPU that can run it fast, like a DGX Spark, you get transcripts with who said what, and nothing leaves your desk! Theres a demo link where you can test and even upload your own audio for transcribing below! Model: huggingface.co/nvidia/NVIDIA… Blog: huggingface.co/blog/nvidia/n… Demo: huggingface.co/spaces/nvidia…
Meet NVIDIA Nemotron 3.5 Transcribe 🎙️ An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint. Watch it keep track of who said what when everyone’s talking at once. 1/ 🧵
4
1
13
1,021
𝗦𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝘀𝗺𝗮𝗹𝗹 𝗟𝗟𝗠𝘀 𝗮𝗿𝗲 𝘄𝗵𝗲𝗿𝗲 𝗶𝘁'𝘀 𝗮𝘁 🔥 Every month, smaller models get more capable, and @liquidai keeps pushing the 𝘀𝘂𝗯-𝟯𝗕 boundary forward. 𝗟𝗙𝗠𝟯 is 6 models across text, vision, audio and omni, all around 𝟮𝗕 or smaller. The VL model claims to compete with 𝗚𝗲𝗺𝗺𝗮 𝟰 𝟭𝟮𝗕 at 𝟭.𝟵𝗕 params. The 1.7B text model has 𝟮𝟱𝟲𝗸 context, and you can fine-tune them on 𝟭 𝗚𝗣𝗨. You don't need a giant model for everything. You need the right small one for the job, running on your own hardware.
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence. > 6 model instances, across text, audio, vision, and omni modalities, generative and decision models. > 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window. > 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding > 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages. > 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language. > 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input. > fine-tune on 1 GPU. > a new tri-hybrid architecture > architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs. > 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters. > 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒 Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device. 1/ 🧵
1
23
1,061
Look at this!! 𝗚𝗲𝗻𝗶𝗲 𝟰 genuinely blows my mind 🤯 Real-time steerable worlds, native audio, characters that talk back, physics that holds up. And it's only a research preview. Now put this next to where 𝗩𝗥 will be in 𝟱–𝟭𝟬 𝘆𝗲𝗮𝗿𝘀. You won't watch a world, you'll step inside one that's generated around you as you move. I can't even picture what that looks like. Incredible work @GoogleDeepMind!! 👏🏼
Excited to share Genie 4 today our most realistic and interactive world model: - steer dynamic environments in real time, shaping events, camera angles, and actions - generate native audio matched to the world, from realistic background sounds to characters who can converse in real time! - simulate photorealistic scenes and real-world physics - create more coherent multi-step events through reasoning from Gemini We've started to roll out as a research preview to trusted testers. Coming soon to Google AI Ultra subscribers in Project Genie and early access partners via the Gemini API and AI Studio.
2
1
1
12
1,382
𝗚𝗟𝗠-𝟱.𝟯-𝗙𝗹𝗮𝘀𝗵 𝗮𝗯𝗹𝗶𝘁𝗲𝗿𝗮𝘁𝗲𝗱 𝗳𝗼𝗿 𝗰𝘆𝗯𝗲𝗿 𝗱𝗲𝗳𝗲𝗻𝘀𝗲 𝗶𝘀 𝘁𝗵𝗲 𝗿𝗶𝗴𝗵𝘁 𝗰𝗮𝗹𝗹 🛡️🔥 𝗙𝗥𝗘𝗘 𝗳𝗼𝗿 𝟰𝟴 𝗵𝗼𝘂𝗿𝘀 on Orca is huge. Go grab it! ━━━━━━━━━━━━ Want a cyber fine-tuned GLM-5.3-Flash running 𝗟𝗢𝗖𝗔𝗟𝗟𝗬? 👇 𝗥𝗘𝗗-𝗦𝗡𝗢𝗪-𝟱.𝟯-𝗙𝗟𝗔𝗦𝗛 by @Blackfrost_AI 🥶❄️ + my SAGE-EXL3 quants: 🟢 2.49 bpw → 𝗢𝗡𝗘 𝗗𝗚𝗫 𝗦𝗣𝗔𝗥𝗞 (97.4 GiB, 262k ctx) 🟣 4.91 bpw → 𝗧𝗪𝗢 𝗗𝗚𝗫 𝗦𝗣𝗔𝗥𝗞𝗦 🎯 𝟮𝟭/𝟮𝟮 on 12 judge-scored cyber tasks 🧵 𝟭𝟱/𝟭𝟱 needle recall out to 240k (independent tests by @yume_arasaki on the one-Spark quant) 🟢 One Spark (2.49 bpw): huggingface.co/vcruz305/RED-… 🟣 Two Sparks (4.91 bpw): huggingface.co/vcruz305/RED-…
Following our initiative of defending with every possible model, we’re launching OrcaCyber GLM-5.3 Flash Abliterated, one of the most powerful OSS models, for all Orca Cyber Tier users. FREE for 48 hours. Limited compute, first come, first served. orcarouter.ai/models/orca/or…
1
6
28
2,332
𝗢𝗽𝗲𝗻 𝘄𝗲𝗶𝗴𝗵𝘁𝘀 + 𝗚𝗲𝗺𝗺𝗮 𝗯𝗮𝘀𝗲 = 𝗶𝘁 𝗿𝘂𝗻𝘀 𝗹𝗼𝗰𝗮𝗹 🔥 Pulling it onto the 4090 tonight to see if it can sit right next to Nanbeige 33B (18.35 GiB, 33.8 tok/s) as a rewrite pass. Local humanizer > paid API humanizer. Results soon 🧪
🚨 Announcing Smaug Humanizer - Humanize your AI-generated text in your tone - open-weights dropped on HF - plug-and-play to humanize your text - sound more human and less robotic - built on Gemma Destroy AI-slop and bring creativity back to the internet
3
3
15
1,239
🚨 @nanbeige 𝗝𝗨𝗦𝗧 𝗗𝗥𝗢𝗣𝗣𝗘𝗗 𝗔 𝗤𝗪𝗘𝗡𝟯.𝟴-𝟮𝟳𝗕 𝗞𝗜𝗟𝗟𝗘𝗥... 𝗔𝗡𝗗 𝗜 𝗙𝗜𝗧 𝗜𝗧 𝗢𝗡 𝗢𝗡𝗘 𝗥𝗧𝗫 𝟰𝟬𝟵𝟬! 🧪🔥 𝗡𝗮𝗻𝗯𝗲𝗶𝗴𝗲𝟰.𝟯-𝟯𝟯𝗕-𝗦𝗔𝗚𝗘-𝗘𝗫𝗟𝟯-𝟰.𝟱𝟬𝗯𝗽𝘄 33 BILLION parameters. DENSE. No MoE shortcuts! And the whole thing runs in 24GB of VRAM! 🤯 📦 𝟭𝟴.𝟯𝟱 𝗚𝗶𝗕 quantized weights 💾 𝟮𝟮.𝟲 𝗚𝗕 measured peak VRAM ⚡ 𝟯𝟯.𝟴 𝘁𝗼𝗸/𝘀 weighted decode 🧠 𝟯𝟲,𝟴𝟲𝟰 context tested with Q4 KV 🏆 𝟴𝟲.𝟲𝟳% 𝗦𝗜𝗫𝗖𝗔𝗧 (104/120) 🎯 𝟵𝟲.𝟲% top-1 agreement vs BF16 (KL 0.0136) 𝗔𝗟𝗟 𝗠𝗘𝗔𝗦𝗨𝗥𝗘𝗗 𝗢𝗡 𝗠𝗬 𝗦𝗜𝗡𝗚𝗟𝗘 𝗥𝗧𝗫 𝟰𝟬𝟵𝟬! 🔥 Weights are PUBLIC and UNGATED 👇 🤗 huggingface.co/vcruz305/Nanb… ━━━━━━━━━━━━━━━━━━━━ 🥊 𝗪𝗛𝗬 𝗜'𝗠 𝗖𝗔𝗟𝗟𝗜𝗡𝗚 𝗜𝗧 𝗔 𝗤𝗪𝗘𝗡 𝟮𝟳𝗕 𝗞𝗜𝗟𝗟𝗘𝗥 The Nanbeige team's own BF16 numbers vs Qwen3.8-27B: 🏆 Terminal-Bench 2.1: 𝟴𝟬.𝟵 vs 73.0 🏆 DeepSWE: 𝟲𝟱.𝟱 vs 42.5 🏆 Automation Bench: 𝟱𝟰.𝟮 vs 38.3 🏆 SkillsBench: 𝟲𝟭.𝟭 vs 56.4 (Original model, not my quant.) So I checked it on my own board. My Qwen3.8-27B runs topped out at 𝟴𝟮.𝟯 (stock Q4) and 𝟴𝟬.𝟬 (AEON Q4+MTP). Nanbeige at 18GB: 𝟴𝟲.𝟲𝟳. 👀 ━━━━━━━━━━━━━━━━━━━━ 🧪 𝗠𝗬 𝗦𝗜𝗫𝗖𝗔𝗧 𝗥𝗘𝗦𝗨𝗟𝗧𝗦 120 questions. Six categories. Strict scoring. Thinking ON. Real Python execution. 🔢 Math: 𝟭𝟬𝟬% 🔥 📋 Instruction Following: 𝟵𝟱% 🔥 🔧 Tools: 𝟵𝟬% 🎯 Truthfulness: 𝟴𝟱% 💻 Code: 𝟴𝟱% 🧠 Knowledge: 𝟲𝟱% 🏆 𝗢𝗩𝗘𝗥𝗔𝗟𝗟: 𝟴𝟲.𝟲𝟳% (𝟭𝟬𝟰/𝟭𝟮𝟬) Transparency: two Code answers ran out of their 32K output budget while still reasoning and were scored as failures. ━━━━━━━━━━━━━━━━━━━━ 📏 𝗣𝗨𝗡𝗖𝗛𝗜𝗡𝗚 𝗪𝗔𝗬 𝗔𝗕𝗢𝗩𝗘 𝗜𝗧𝗦 𝗪𝗘𝗜𝗚𝗛𝗧 🟣 𝗤𝘄𝗲𝗻𝟯.𝟴-𝗙𝗹𝗮𝘀𝗵-𝗡𝗲𝘅𝘁 (125B MoE, 𝟯.𝟴𝘅 𝗯𝗶𝗴𝗴𝗲𝗿): my Q4_K_M GGUF scored 𝟴𝟵.𝟮. Nanbeige is only 𝟮.𝟱 𝗽𝗼𝗶𝗻𝘁𝘀 behind and ties my 3.05bpw EXL3 (𝟴𝟲.𝟳). 🔵 𝗚𝗟𝗠-𝟱.𝟯-𝗙𝗹𝗮𝘀𝗵 (320B MoE, ~𝟭𝟬𝘅 𝗯𝗶𝗴𝗴𝗲𝗿): 𝟵𝟭.𝟳 unquantized via API. Nanbeige is within 𝟱 𝗽𝗼𝗶𝗻𝘁𝘀 on a gaming card. 🤯 Different Sixcat versions and serving setups, so not a controlled ranking. But that's a dense 33B on a 4090. ━━━━━━━━━━━━━━━━━━━━ ⚡ 𝗧𝗛𝗘 𝗥𝗘𝗟𝗘𝗔𝗦𝗘 ✅ ExLlamaV3 + TabbyAPI (OpenAI-compatible) ✅ Thinking + tool calling ✅ Mixed-precision SAGE-EXL3, exact allocation published ✅ GitHub recipe with checksums and full test results The original advertises 512K context; my 4090 qualification covers 36,864 tokens. No DGX Spark. No rented datacenter. Just a 24GB card. 🚀 🛠️ 𝗥𝗘𝗖𝗜𝗣𝗘 github.com/vcruz305/Nanbeige… 💚 Congrats to the Nanbeige team on an impressive release! 𝗧𝗛𝗘 𝟮𝟰𝗚𝗕 𝗚𝗣𝗨 𝗖𝗟𝗔𝗦𝗦 𝗝𝗨𝗦𝗧 𝗚𝗢𝗧 𝗔 𝗦𝗘𝗥𝗜𝗢𝗨𝗦 𝗡𝗘𝗪 𝗔𝗚𝗘𝗡𝗧𝗜𝗖 𝗠𝗢𝗗𝗘𝗟. 🔥
🚀Excited to announce Nanbeige4.3-33B! To push the performance ceiling of small agents, our strategy in Nanbeige4.3 starts with building a powerful teacher first. Nanbeige4.3-33B delivers SOTA agentic performance among open models at ~30B parameters: * Strong results on agentic coding and cowork benchmarks. * When integrated into mainstream agent harnesses, it tackles real-world tasks with cost-efficient inference. * It will serve as a teacher for upcoming smaller Nanbeige models, starting with Nanbeige4.3-3B. huggingface.co/Nanbeige/Nanb… huggingface.co/Nanbeige/Nanb… #LLM #Agent
8
4
1
27
2,861
Qwen3.8-Flash-Next on one DGX Spark: @MiaAI_lab's TensorFold recipe decodes code at 43.2 tok/s, @ViC305's two-model EXL3 recipe at 58.4 tok/s. Setup: – Same voxel-world prompt, both as an agent in Qwen Code 0.25.2, one prompt, one try, no help, thinking on – Context: 262,144 tokens – Mia's recipe: NVFP4 with MTP on @ashxhart's TensorFold, 5 slots, answer limit 131,072 tokens – ViC305's recipe: EXL3 3.05 bpw on ExLlamaV3 and TabbyAPI, 1 slot, answer limit 262,144 tokens, MiniCPM 4.7 loaded beside it Mia's recipe, world Aetherfall: – Decode, code: 43.2 tok/s – 500 words: about 16.9 s – 8 requests at once: 87.6 tok/s in total – Reading a 32K-token prompt: 2,163 tok/s – Memory at 262,144 tokens of context: 87.01 GiB – Agent time: 108.6 min – Browser check: 8.5/10 ViC305's recipe, world AETHERFALL: – Decode, code: 58.4 tok/s – 500 words: about 12.6 s – 8 requests at once: 41.5 tok/s in total – Reading a 32K-token prompt: 662 tok/s – Memory with both models loaded, 125,892-token prompt: 87.0 GiB – Agent time: 60.9 min – Browser check: 8.5/10 Quality: – MMLU-Pro: 68.3% against 67.3% – LiveCodeBench: 62.5% against 65.0% – GPQA with thinking: 70.0% for both (30 questions, ± 15.6 points) Mia's recipe is the stronger setup for parallel work and long prompts, and I like its world better. ViC305's decodes a single request about 35% faster. Good to know: – Different quants and engines, and different answer limits, so the agent times do not compare the models alone. – Mia's 8-at-once total comes from a second Spark. The author's README shows 88.6 tok/s for code; my fixed code prompt is mostly prose. – ViC305's memory number comes from a test up to a 125,892-token prompt, Mia's from the full 262,144. Play both worlds in your browser. All numbers, the exact command and the raw output: Mia's recipe: niklaslenz.com/v2/qwen38-fla… ViC305's recipe: niklaslenz.com/v2/qwen38-fla… Recipe by @MiaAI_lab on @ashxhart's TensorFold, model by @Alibaba_Qwen. Recipe, MiniCPM quant and engine forks by @ViC305, EXL3 quant by @turboderp_.
9
1
24
1,790
Amazing work by @wikiwayne ! Im extremely glad to see my EXL3 quants take the top 3! Especially this 1.62bit Deepseek v4.1 Flash EXL3 beating my lossless Qwen Flash Next EXL3!! 👀 love the site layout btw great job on testing so many models!!
Ran locally on my GX10: DeepSeek V4.1 Flash at 1.62 bits (vcruz305 pack) is ranked first of 34 rows on WikiBench. 5.35 out of 10. Coding 7.75, research 9.3. Nobody on the board reaches 6. Did you expect a 1.62-bit quant to lead?
5
1
28
2,000
🔥 MiniMax M3.1 Flash (428B) on 2 NVIDIA DGX Sparks, TP2 over RoCE 🧠 Quant by @ViC305 (SAGE EXL3 2.15 bpw) 🙌 📏 300K context, 726K KV pool 🏎️ Up to 39 tok/s single stream, 65-78 tok/s at 4 users 🛠️ 2 vLLM-EXL3 patches for TP2 Recipe + benchmarks 👇 github.com/tonyd2wild/MiniMa…
11
6
46
3,187