Local AI on 2x DGX Spark. Inference engineering. Frontier open models, optimised. Pull-and-run recipes and real benchmarks. Next up: M5 Max vs Spark.
Sydney, New South Wales
Joined July 2026
- Tweets1.2K
- Following116
- Followers143
- Likes73
Pinned Tweet
New Kolibri-1 recipe built for 2x DGX Spark on TensorFold github.com/ajensenwaud/Kolib… @MiaAI_lab @ashxhart
Anders retweeted
TensorFold engine 1.2.1 is out.
A few highlights:
• MiniMax-M3.1-Flash on CUDA across two GB10s, using Mia-AiLab's EXL3 pack.
• Gemma 4 31B on GB10 with Google's MTP assistant: 28.3 tok/s versus 12.2 undrafted, at one stream.
• Qwen3.5/3.6 windows of 1,048,576 tokens on CUDA with YaRN.
• MOSS-Image end to end on Metal, via the CLI.
• Gemma 4 E4B prompt processing 2 to 2.6 times faster on Intel Arc Pro B70.
There are also opt-in GLM long-conversation improvements, server load reporting, Living Weights bulk fact deletion and fixes for lesson failures and Flash Next's long-prompt slowdown on GB10.
Huge thanks to everyone who contributed, tested and reported issues:
@andersjw_ @atasrb @Bizuayeu
@grantoverton @ivanfioravanti @jayleaton
@niklaslenz_ai @2Halves1brain @wesley4546
And on GitHub: AbrahamCete, akol1, BlivionIaG, BobClawblaw, gilby, Jenpo, jschmied, kieranscotney, laudney, Mirrdhyn, ppardus, russellmm and ThomasHalwax.
Upstream fix credits: florianbrede-ayet and mikolaj92.
Thank you for the time, code, testing and hardware access you keep sharing with TensorFold. Apologies if I've missed anyone.
@petruspennanen @plotarmordev @MiaAI_lab @volatilemarkts
tensorfold.dev
Small change but now officially contributing to a super important project (@tensorfold_dev) @ashxhart
Whilst my PRs for Muse Spark and Mimo are waiting to be integrated into @tensorfold_dev, I have started quantising and porting Ling 3.1-flash to 2 x DGX Spark / EXL3 using TensorFold. @AntLingAGI Watch this space!
Compiled the top five recipes for DeepSeek 4.1 1017 running on 2x DGX Spark. Leader is 113 tok/s. All run on TensorFold. Numbers based on author claims, not my own benchmarking.
1. @Blackwellboy, 113 tok/s
github.com/Blackwellboy/Deep…
2. @bertholomusai, 112.6 tok/s
github.com/bertholomus/deeps…
3. @MiaAI_lab, 109.9 tok/s
github.com/MiaAI-Lab/DeepSee…
4. @jayleaton, 87.9 tok/s
github.com/jayleaton/deepsee…
5. @Tech2Wild, 73.1 tok/s
github.com/tonyd2wild/DeepSe…
Methods differ between recipes, so treat gaps under 5% as noise.
Very proud to have contributed to this release! @ashxhart
(GH: ajensenwaud)
TensorFold 1.1.6 is out.
Eight chats arrive together with the same new 28k-token system prompt. On a GB10 with Qwen3.8-27B, the last chat gets its first token after 31s instead of 201s. Every reply matches its run alone.
Also in this release:
- Opt-in conversation disk cache on Mac.
- Qwen3.8-27B on Metal keeps decoding streams moving during long prompts.
- Gemma 4 CUDA chat-cache fixes and GLM-5.3-Flash image-cache reuse on Metal.
- Spark-X2.5/2.6-4B on CUDA and Metal.
- Drama speech sampling controls and speaker tags.
- TensorFold plan estimates CUDA model memory before downloading or loading weights.
There are also tool-call, memory-admission and macOS download fixes, plus updates to the experimental DeepSeek-V4.1 engine.
Huge thanks to the contributors:
@jvr0x @bobclawblaw @krunkosaurus @lasse_ai
And on GitHub: jschmied, wojo, ecohash-co, ajensenwaud, akol1, philip-pentatonic, gilby and BlivionIaG.
Thanks to Boscoeuk on GitHub for the reports too.
To everyone testing, sharing hardware access and supporting TensorFold, thank you. If I've missed anyone, I'm sorry.
@MiaAI_lab @volatilemarkts @plotarmordev @petruspennanen
tensorfold.dev
An 816B model, Muse Spark, running on 2x DGX Spark (1.8bpw), engineering the USS Enterprise. Attached is a nice video of its handling of Three.js, using @NousResearch Hermes Agent as the harness. Link to the recipe in the comments.
Meta's 816B Muse Spark Open is 1600 GB at 16bpw. It now runs on two in 195 GB, 4x faster than Unsloth's 2bpw model with 1.8bpw.
Unsloth's version does 7 tok/s on the same two boxes. Muspark (built on TensorFold) does ~32tok/s with a ~900K context.
Quality unchanged at GSM8K 196/200 vs 193/200.
Recipe and benchmark in the replies.
Recipe, patches and benchmarks are open:
github.com/ajensenwaud/muspa…
The remarkable thing is Muspark runs reasonably well without loss of quality compared to baseline - 3-4x throughout against Unsloth. Big driver was thoughtful quantisation (variable) and TensorFold.
sparkDash is awesome. I use it to monitor my Sparks at home. As a former Unix system admin, I can highly recommend it. nitter.gq/miaai_lab/status/21131…
DeepSeek V4.1 Flash decode speeds posted on X in the last 48 hours. All figures are self-reported and the setups differ, so this isn't a like-for-like comparison.
410 tok/s, 4x RTX PRO 6000, engine not stated, @0xSero
~320 tok/s, 8x RTX PRO 6000 Max-Q, vLLM + DSpark TP8, @hshimodaira nitter.gq/hshimodaira/status/211…
~200 tok/s, 4x RTX PRO 6000, vLLM with the rtx6kpro recipe, @siegl_rene nitter.gq/siegl_rene/status/2112…
113 tok/s, 2x DGX Spark, Blackfold, @Blackwellboy nitter.gq/Blackwellboy/status/21…
79 tok/s, M3 Ultra, engine not stated, @dataphwj nitter.gq/dataphwj/status/211247…
70-100 tok/s, 2x DGX Spark, MiaAI-Lab recipe + knapcio TP4 fork, @unsignedbytes nitter.gq/unsignedbytes/status/2…
68 tok/s, M3 Ultra 512GB, oMLX with MTP, @jundotkim nitter.gq/jundotkim/status/21128…
68-71 tok/s, 2x DGX Spark, EXL3 2.9 bpw, @easthambuilds nitter.gq/easthambuilds/status/2…
48.5 tok/s, 2x M3 Ultra, ds4 MXFP4, @ivanfioravanti nitter.gq/ivanfioravanti/status/…
40-50 tok/s, 2x DGX Spark, vLLM EXL3 2.7 bpw, @sfxnz nitter.gq/sfxnz/status/211219585…
Next up: 2x DGX Spark on NVIDIA Flare. Numbers to follow.
DeepSeek-V4.1-Flash-1017 on 2x DGX spark, quantised in-house to 2.7 bpw EXL3 🔥
- 40-50 tok/s decode
- 1M context
- Vision on vLLM
- KL 0.025 vs the unquantised model
This recipe will be significantly improved over time!
./run.sh and go
github.com/sfxnz/DeepSeek-V4…
Anders retweeted
Do you know the meaning of the @NousResearch logo?
The girl is called the "Nous Girl". She was drawn in July 2023 by Nous's first artist, who goes by John Galt. Yes, the Atlas Shrugged rebel
She stands for the rebellion of decentralized AI, and for keeping our humanity in the face of censorship and machine intelligence
The words around her: "Rebellion to tyrants is obedience to God."
That line goes back to 1776. Jefferson, Franklin and Adams put it on their proposed Great Seal of the United States. Congress said no, so Jefferson put it on his own personal seal instead
Nous pointed it at AI. Today's tyrants are closed labs and gatekeepers deciding who gets access to intelligence. The rebellion is open weights, open agents and open infrastructure.
That's why I build on Hermes
Wearing it proudly 🏴
Anders retweeted
We have 2700+ signatures on our petition to protect your right to intelligence.
Sign: righttointelligence.org
Reading this article about companies restricting tokens and AI usage makes me appreciate owning my own intelligence.
theguardian.com/technology/2…
Apple research: Muse Spark Open running on a cluster of 4 Mac Studios at 36 tok/sec
machinelearning.apple.com/re…
