Pinned Tweet
What fits on one DGX Spark, and what needs two. Only models I ran myself, one user, real prompts.
One Spark: Qwen3.8-Flash-Next at 66 tok/s on JSON. Nemotron 3.5 Bolt at 96 with its draft model. A dense 27B at 11.
Two Sparks: DeepSeek-V4.1-Flash (763B) at 103 tok/s, but only 7-9 GiB stays free on each Spark. MiniMax-M3.1-Flash (428B) at 27-29 with 2.15 bpw, 20-22 with 3.06 bpw. Qwen3.8-Flash-Next goes from 66 to 99.
Two OCR models on one RTX 4090 read 755 pages of OmniDocBench with almost the same score: MiniCPM-OCR 1.0 in 6.4 minutes, Xiaomi-OCR-0 in 10.8.
Overall score on the 755 English pages of OmniDocBench v1.6: Xiaomi-OCR-0 95.2, MiniCPM-OCR 1.0 94.9. MiniCPM ran at 1.97 pages/s, Xiaomi at 1.17.
Each model ran with its makers' pipeline on one vLLM nightly. Xiaomi finds the page layout with PaddleX first, then reads each region.
The picture is a page of NIST FIPS 197, the AES standard. Both read the matrices and formulas right. MiniCPM puts the equation numbers on their own lines.
Qwen3.8-Flash-Next on one DGX Spark built this live monitor for my two Sparks in 6 minutes.
The setup: NVIDIA's FP4 build on vLLM with MTP, opencode 2.2.5 with all permissions on, inside a Docker sandbox, thinking off. It wrote a FastAPI backend with tests and a React app with Vite, and kept the dev server running, so the page changed after each edit.
17.5K tokens at about 50 tok/s while writing. The video runs in real time for 12 s in the middle; the rest is sped up 8x.
One mistake: it labels the link 100 GbE. The QSFP link between the two Sparks is 200 Gb/s.
I came across this one while browsing Hugging Face.
RFD4-Proteína designs new proteins on one DGX Spark: a binder for PD-L1 in 16 s.
It designs the sequence and the full 3D structure together, from a target protein, a motif or a small molecule. Seven of the nine examples on the model card ran as they are, at 3 to 46 s a design.
The colored chains are the new designs, grey is the target. These are computer designs, not lab results.
It needs only 1.5 GB of GPU memory, so one Spark is far more than enough. The card's H200 time is about 7x faster.
NVIDIA's 120B vision-language model on one DGX Spark: 71 tok/s on JSON, 64 on code.
Nemotron 3.5 Super VL, NVIDIA's NVFP4 build with its LiLiCorr draft, on vLLM. The draft guesses 7 tokens ahead and the model checks them in one step: 6-7 land on structured text, about 3 on prose.
At 128K context it still decodes 29 tok/s, and it reads a document page image and answers in 5.5 s.
Quality with thinking off: GPQA Diamond 78% and IFEval 82%. GLM-5.3-Flash FP4 on two Sparks scored 71% on the same GPQA set.
NVIDIA's Spark command needed two changes to keep 10+ GB free: 0.70 of the memory instead of 0.75, and a 256K window instead of 1M.
A strong model for one Spark, and it reads images too.
A 65 s scene from Charade (1963), transcribed in 5 s on one DGX Spark, with every word timed and both speakers kept apart.
NVIDIA's Nemotron 3.5 Transcribe, FP8 build, on vLLM. It runs offline: the whole clip at once, then the words are synced to playback.
New since my BF16 test: the FP8 build keeps the same accuracy at 1.5x the speed. An hour of earnings calls (plain transcript) now takes about 2 minutes.
A solid open ASR model. Subtitles with speakers and word timings from one local request, and nothing leaves the box.
Meet NVIDIA Nemotron 3.5 Transcribe 🎙️
An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint.
Watch it keep track of who said what when everyone’s talking at once.
1/ 🧵
Recipe, FP8 and NVFP4: github.com/atakann/dgx-spark…
Its client writes .srt subtitles with speaker tags.
Both, on my two DGX Sparks. Here is GLM-5.3-Flash: @MiaAI_lab's EXL3 4 bpw build and TensorFold recipe, against NVIDIA's FP4-Local on TensorFold 1.1.9.
One user, tok/s (EXL3 vs FP4):
JSON 82 vs 52
code 83 vs 47
rewrite 73 vs 52
explain 58 vs 42
story 46 vs 36
Four users: 82 vs 56 tok/s in total. At 128K context it still decodes 47 tok/s.
Quality with thinking off: GPQA Diamond 69% vs 71%, about even. IFEval 81% vs 64% on the same 200 prompts.
The fastest GLM-5.3-Flash I have measured on two Sparks. Looking forward to the updates.
Mia's recipe, the version I ran (bf5f945): github.com/MiaAI-Lab/GLM-5.3…. One tip for the first start: its CUDA extensions build after the memory budget is taken and need 6-8 GiB more, so build them once with no model loaded.
Qwen3.8-Flash-Next on one DGX Spark against LFM3-VL-2B, on every aerial tile of Tucson.
The same 13,804 tiles and the same plain-words prompt. Flash-Next runs NVIDIA's FP4-Local weights on vLLM.
Planes counted:
Flash-Next 5,339 in 3 h 28 min
LFM3-VL-2B 3,314 in 69 min
Not all of Flash-Next's boxes are planes. The strip near the end shows roofs and cars.
On DOTA's validation set, planes found / wrong boxes:
Flash-Next 99% / 26%
LFM3-VL-2B 89% / 18%
Ships, where the small VLM struggles: Flash-Next finds 82%, LFM 30%.
Models: @QwenDevs Qwen3.8-Flash-Next, @NVIDIAAI FP4-Local, @liquidai LFM3-VL-2B. Recipe link below.
Every plane in Tucson, AZ, on one RTX 4090.
13,804 aerial tiles, NAIP 2023 at 0.3 m per pixel.
YOLO26m-OBB: 2,827 planes in 2 min 12 s.
LFM3-VL-2B: 3,314 planes in 69 min.
The VLM's higher count is not a win. Most of its
extra planes are helicopters, which YOLO's DOTA
classes keep apart.
On DOTA's validation set, YOLO finds 99% of planes
with 4% of its boxes wrong. LFM finds 89%, with 18%
wrong.
Two kinds of model:
YOLO26m-OBB is a detector trained on DOTA's labeled
aerial photos. It knows 15 classes and nothing else.
LFM3-VL-2B is a general vision-language model. You
ask it in plain words for any target, no training.
For planes, the trained detector wins, 31x faster.
Still, the VLM was never fine-tuned for this. From
one plain-words prompt, a 2B model finds 89% of the
planes. For targets no detector knows, that's useful.
Models: @ultralytics YOLO26m-OBB and @LiquidAI_ LFM3-VL-2B.
How it ran: one DGX Spark, vLLM, 8 requests at once, MTP 4, thinking off. The GPU was 90% busy, and the lowest free memory was 12.0 GiB. Recipe: github.com/atakann/dgx-spark…
Every plane in Tucson, AZ, on one RTX 4090.
13,804 aerial tiles, NAIP 2023 at 0.3 m per pixel.
YOLO26m-OBB: 2,827 planes in 2 min 12 s.
LFM3-VL-2B: 3,314 planes in 69 min.
The VLM's higher count is not a win. Most of its
extra planes are helicopters, which YOLO's DOTA
classes keep apart.
On DOTA's validation set, YOLO finds 99% of planes
with 4% of its boxes wrong. LFM finds 89%, with 18%
wrong.
Two kinds of model:
YOLO26m-OBB is a detector trained on DOTA's labeled
aerial photos. It knows 15 classes and nothing else.
LFM3-VL-2B is a general vision-language model. You
ask it in plain words for any target, no training.
For planes, the trained detector wins, 31x faster.
Still, the VLM was never fine-tuned for this. From
one plain-words prompt, a 2B model finds 89% of the
planes. For targets no detector knows, that's useful.
Models: @ultralytics YOLO26m-OBB and @LiquidAI_ LFM3-VL-2B.
How it ran: 192 m tiles from Microsoft Planetary Computer. YOLO26m-OBB with Ultralytics' DOTAv1 weights. LFM3-VL-2B Q8_0 on llama.cpp, thinking on. Planes cut by tile edges are merged. The GPU was 65% busy for YOLO and only 39% for LFM: CPU-side sampling limits the VLM.
GLM-5.3-Flash on two DGX Sparks with NVIDIA's
FP4-Local weights: TensorFold is up to 1.5x faster
than vLLM for one user, and level at four.
One user, real prompts, tok/s (TensorFold vs vLLM):
JSON 51.5 vs 50.3
code 47.1 vs 41.0
rewrite 52.0 vs 40.7
explain 42.0 vs 33.3
story 36.3 vs 24.6
A 128K prompt reads in 91 s, and turn 2
comes from the cache in 1.3 s.
This is the stock TensorFold release, no patches.
My first PR to @ashxhart's TensorFold landed
this week, a small one. Happy to keep helping improve it.
GLM-5.3-Flash on two DGX Sparks with @NVIDIAAI's FP4-Local weights (TP=2): 31 tok/s for one user, 55 for four.
Decode, total tok/s (thinking off, 200-token prompts, 150 tokens out):
1 user 31.2
2 users 45.3
4 users 54.7
4 users at once is the server limit (max-num-seqs 4).
An 8k-token prompt reads in 3.6 s.
Engine: vLLM 8244d1c (eugr/spark-vllm-b12x nightly-20261015), the model card's command with a few changes. The important one: PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True on both nodes (from Bizuayeu's report in discussion #7 on the HF card). Without it, free memory fell to 4 GB right after the weights loaded. With it, the low point was 12 GB during the load and about 20 GB stays free while serving.
Recipe, one command on two Sparks:
github.com/atakann/dgx-spark…
Built on TensorFold's own GLM recipe:
github.com/ashhart/TensorFol…
One DGX Spark detecting and tracking 16 1080p cameras, every frame, 31-32 FPS per camera.
YOLO26s on DeepStream 9.2, TensorRT FP16.
One Spark tops out near 18 cameras at 30 FPS.
Clips from Pexels. Recipe in the reply.
Run thread-safe inference with Ultralytics YOLO26! ⚡
Avoid race conditions in multi-threaded workflows by creating a separate model inside each thread, or use ThreadingLocked when you need safe shared-model access. Perfect for multi-stream and real-time monitoring systems.
Learn more ➡️ docs.ultralytics.com/guides/…
#Ultralytics #Threading #MachineLearning
Recipe: github.com/atakann/dgx-spark… (vision/yolo26-deepstream)
NVDEC decode, batch of 16, NvSORT tracker.
16 streams: 33.4 FPS each, 535 in total.
GPU 96%, NVDEC 34%: compute is the limit.
