Run Thinking Machines Lab's Inkling 2 Mini locally on AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151)
huggingface.co/jcbtc/Inklingโฆ
25
Ivan Rocha retweeted
huggingface.co/jcbtc/Inklingโฆ
Here's the link to the Nvidia version you can also find the AMD version on my hugging face page
Inkling 2 Mini on one DGX Spark, from Thinking Machines' own 4-bit weights with no quantization of mine: 65 tok/s on a long coding answer. In a chat it feels instant, the first token comes after 0.2 s.
The right side shows speculative decoding live. A small draft head guesses up to 8 tokens and the model checks them in one pass, so boilerplate code goes through 8 at a time and plain English 1 to 3.
3
18
1,060
Ivan Rocha retweeted
It is becoming f*cking dangerous: your agent can now take any video apart shot by shot, tell you exactly how it was made, and rebuild the whole edit on your own machine - 3 free tools, 28k stars combined
rea-video + ffmpeg-skill + pixelfx...
1. Pick a video you want to understand - yours, or any you're allowed to study
2. Install rea-video: tell your agent "install github.com/JuneYaooo/rea-vidโฆ, then break down this video: clip.mp4". it writes VIDEO.md - every shot labeled filmed, AI-generated or coded, with the exact setup to remake it
3. Install ffmpeg-skill: npx ffmpeg-skill setup. now the agent cuts, captions, reframes and exports locally - 42 ffmpeg tools, no cloud, no keys
4. Install pixelfx: npm i github:patch-gg/pixelfx. ask for any of 244 effects, or have it write you a new one
5. Say "rebuild this edit with my footage, same beats" - and it does, shot for shot
rea-video tells you how it was made. ffmpeg-skill rebuilds it. pixelfx styles it.
my .claude/ folder with all three wired in is in the reply below
Replying to @filicroval
solid speed, iโm getting 47 tok/s but with 1M context
github.com/thefullctx/inklinโฆ
Ivan Rocha retweeted
๐งช Inkling 2 Mini on ONE DGX Spark at the full 1M context โก
๐ง Official NVFP4 + DSpark in vLLM
๐ฆ 3.3M-token KV pool (FP8 KV)
๐ฏ 950K-token needle test: passed โ
๐ ~47 tok/s single user
Recipe + launch scripts ๐
github.com/tonyd2wild/Inklinโฆ
Ivan Rocha retweeted
Weโre proud to partner with Thinking Machines Lab to bring Inkling 2 Mini to Llama day-0 ๐ค
Weโre releasing 4-bit and 8-bit versions of the model for your local setups, which pack a ton of intelligence for consumer hardware.
Ivan Rocha retweeted
I highly recommend running this skill on your projects
vulnerability-scanner reviews your code for security vulnerabilities, including AI agents and MCP servers
What it does:
- Maps the attack surface and trust boundaries.
- Detects prompt injection and insecure agent tools
- Checks for exposed secrets and vulnerable dependencies
- Reviews MCP servers, skills, hooks, and CLAUDE.md files
- Identifies OWASP vulnerabilities and CI/CD security risks
- Verifies findings to reduce false positives
- Reports each issue with Attack, Impact, and Fix
Some results from testing:
- Automatically activated in 7/7 runs
- 0 false positives across 7 decoys
- Compatible with 6 coding agents
IMPORTANT: Itโs read-only and never modifies your code
Install:โจnpx claude-code-templates@latest --skill security/vulnerability-scanner
aitmpl.com/component/skills/โฆ
Ivan Rocha retweeted
You can now Run locally Uncensored Inkling-2-Mini on 16GB.
- it beating Qwen3.8-27B
- refusal direction removed from the residual stream.
- Same looped architecture.
- LoRA adapter + ready GGUFs coming..
- huggingface.co/AtomicChat/Inโฆ
You can now run a 26B model on a laptop that beats much larger models on coding and agentic work.
the model crown for its size on coding benchmarks.
- Inkling 2 Mini (5.4B active MoE, looped transformer)
- supports 1M context,
- scores 48.2% on DeepSWE v1.1
- and 77.9% on SWE-bench Pro (Hard)
ready for Unsloth + llama.cpp.
- huggingface.co/unsloth/Inkliโฆ
How to increase Qwen3.8-27B inference speed from 25 tok/s to 91 tok/s for free on the same GPU?
The authors of the inference-optimisation-skill project demonstrated this using the Intel Arc Pro B70 and Qwen3.8-27B as an example:
- llama.cpp Q4_K_M: 25 tok/s
- vLLM EXL3 4 bpw with custom kernels: 91.2 tok/s
They also created an open-source skill that allows an AI agent to independently search for ways to optimize local models.
How does it work?
- calculates the theoretical hardware performance limit
- analyzes what limits the model's speed
- changes one parameter and runs a benchmark
- Compares the results with control measurements
- rejects changes that degrade stability or performance with long context
Results on RTX 3090 with Qwen3.8-27B in EXL3 3 bpw and MTP:
- Prose: 96.2 tok/s
- Code: 141.1 tok/s
This is not another inference engine, but an optimization methodology that can be passed to an AI agent.
Would you let an AI agent optimize the configuration of your local LLM all night long?
github.com/sybil-solutions/iโฆ
Ivan Rocha retweeted
Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM.
Thinking Machinesโ Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size.
Run on Unsloth, llama.cpp
GGUF: huggingface.co/unsloth/Inkliโฆ
Guide: unsloth.ai/docs/models/inkliโฆ
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.
thinkingmachines.ai/news/inkโฆ
Ivan Rocha retweeted
Inkling 2 Mini from Thinking Machines! ๐ฅ
- 26B total params, 3.4B active (5.4B looped)
- Weights just 20 GB
- Small enough to run on a laptop
- Ahead in 10 of 13 in coding vs Qwen3.8-27B.
I'll try it out on my dgx spark.
Link below.
Ivan Rocha retweeted
Maya v1.1.69 is out: Hy4-Flash gets faster on every kind of PC. โก
โ
Drafting on 8-12 GB cards with a small DFly2 drafter (up to +29% on an RTX 3050 8 GB)
โ
AMD / Strix & Gorgon Halo: prompts 2-2.5x faster, drafting ~30% faster
โ
Long prompts on NVIDIA: up to +56% at 128K
โ
Contexts up to 1M on ordinary cards, with the cache in RAM
โ
Pictures at full size, even on 8 GB cards
๐งช Hy4-Flash on Windows + AMD, experimental
Drafting still checks every token. Thanks to @needmorevram, @superalesha and every contributor.
github.com/mw00/project-mayaโฆ
Ivan Rocha retweeted
Big update for GLM 5.3 Flash for 2x DGX Sparks
Running latest TensorFold (v1.3.0). Lots of changes:
- Faster decode: CUDA kernels now chain directly with fewer launch gaps (PDL)
- Cleaner stack: Our paged FP8 KV, MoE, and DFlash2 speedups are now built into upstream (100+ patches down to 13)
- Smarter prompt cache: Cache space is reserved up front, and multi-turn thinking chats reuse cached reasoning instead of re-reading history
- More stable 2-GPU vision: Long image prompts no longer time out the second GPU
Even more changes and improvements coming soon!
Get it here:
github.com/MiaAI-Lab/GLM-5.3โฆ
Ivan Rocha retweeted
Tencent's 220B Hy4-Flash now runs on a home PC with an 8 GB graphics card. ๐
Maya v1.1.68 is out, with four quants we made ourselves for @TencentHunyuan's new model. Each one is measured against the full-precision original:
Maya-XS ยท 45 GB ยท 88.5% same top token
Maya-S ยท 53 GB ยท 91.8%
Maya-M ยท 68 GB ยท 93.6%
Maya-L ยท 90 GB ยท 96.6%
How does 220B fit? What doesn't fit the graphics card runs on the CPU from RAM, what doesn't fit RAM streams from the SSD, and the model's 63 GB n-gram table stays on the SSD.
โ
Drafting: the model checks every drafted token
โ
Vision, thinking on or off, tool calls
โ
NVIDIA RTX 20 series and newer on Linux and Windows
โ
AMD on Linux, Strix Halo and Gorgon Halo included
โ
One setup command picks the size and context that fit your PC
Which one will you pick?
Full notes ๐
github.com/mw00/project-mayaโฆ
๐จ๐ฅ ๐ง๐๐ก๐๐๐ก๐ง ๐๐ฌ๐ฐ-๐๐๐๐ฆ๐ ๐ฆ๐๐๐-๐ฅ๐๐ซ๐๐ฏ ๐ฐ.๐ด๐ฐ ๐๐ฃ๐ช, ๐ต๐ฒ.๐ต๐ฎ% ๐ง๐ข๐ฃ-๐ญ ๐๐๐ฅ๐๐๐ ๐๐ก๐ง, ๐๐ก๐ ๐ฆ๐๐ญ๐๐ ๐๐ข๐ฅ ๐ข๐ก๐ ๐๐๐ซ ๐ฆ๐ฃ๐๐ฅ๐! ๐คฏ๐
The fidelity results are looking GREAT!
๐ ๐ต๐ฒ.๐ต๐ฎ% ๐ง๐ข๐ฃ-๐ญ ๐๐๐ฅ๐๐๐ ๐๐ก๐ง
๐ฌ ๐ฌ.๐ฌ๐ฎ๐ด๐ด๐ญ๐ณ ๐ ๐๐๐ก ๐๐ ๐๐๐ฉ๐๐ฅ๐๐๐ก๐๐
๐พ ๐ต๐ฎ.๐ต ๐๐ถ๐ ๐๐ฃ๐จ-๐ฅ๐๐ฆ๐๐๐๐ก๐ง ๐ช๐๐๐๐๐ง๐ฆ
I just finished running @TencentHunyuanโs newly released Hy4-Flash through my latest SAGE-REXL3 quantization process!
๐ง๐๐๐ฆ ๐๐ฆ ๐ ๐ฎ๐ฎ๐ฌ๐ ๐ฃ๐๐ฅ๐๐ ๐๐ง๐๐ฅ ๐ ๐ข๐! ๐ง
โก ~12B active parameters per token
๐ง Native 1M context architecture
๐ Built-in 5-layer DFly2 speculative drafter
๐ป Hybrid sparse/sliding-window attention
๐ฆ 4.84 bpw mixed-precision EXL3
And hereโs the interesting partโฆ
I preserved critical attention, routing and normalization components at higher precision while compressing the rest with SAGE.
The result?
๐ ๐ต๐ฒ.๐ต๐ฎ% ๐ก๐๐ซ๐ง-๐ง๐ข๐๐๐ก ๐๐๐ฅ๐๐๐ ๐๐ก๐ง ๐ช๐๐ง๐ ๐ง๐๐ก๐๐๐ก๐งโ๐ฆ ๐ข๐ฅ๐๐๐๐ก๐๐ ๐๐๐ญ๐ฒ ๐๐๐๐๐๐ฃ๐ข๐๐ก๐ง! ๐ฅ
โโโโโโโโโโโโโโโโโโ
๐ฅ๏ธ ๐๐๐ฆ๐๐๐ก๐๐ ๐๐ข๐ฅ ๐ข๐ก๐ ๐๐๐ซ ๐ฆ๐ฃ๐๐ฅ๐!
The active model weights occupy ~92.9 GiB, while the additional 60.24 GiB Engram lookup tables are stored in NVMe-backed memory-mapped files.
Thatโs the memory layout Iโm targeting for single-Spark inference. Full serving and context validation are still ahead.
๐๐ก๐ ๐ฌ๐๐ฆโฆ ๐ ๐๐จ๐๐ ๐ฅ๐๐ฃ๐ฅ๐ข๐๐จ๐๐๐๐๐ ๐๐ก๐๐๐ฅ๐๐ก๐๐ ๐ฅ๐๐๐๐ฃ๐ ๐๐ฆ ๐ข๐ก ๐๐ง๐ฆ ๐ช๐๐ฌ! ๐๐ฅ
๐ค ๐ ๐ข๐๐๐ ๐ช๐๐๐๐๐ง๐ฆ:
huggingface.co/vcruz305/Hy4-โฆ
Huge credit to @TencentHunyuan and @tencentcloud for bringing Hy4-Flash to the open-weight community! ๐
๐๐ฌ๐ฐ-๐๐๐๐ฆ๐. ๐ฐ.๐ด๐ฐ ๐๐ฃ๐ช. ๐ต๐ฒ.๐ต๐ฎ% ๐๐๐ฅ๐๐๐ ๐๐ก๐ง.
๐ง๐๐ ๐ฆ๐๐๐ ๐ค๐จ๐๐ก๐ง๐๐ญ๐๐ง๐๐ข๐ก ๐๐ข๐จ๐ฅ๐ก๐๐ฌ ๐๐ข๐ก๐ง๐๐ก๐จ๐๐ฆ! ๐๐ฅ
Hy4 Flash is now released on huggingface!! ๐๐๐
Thanks to @aitrackerbot always being on point! You can finally run Haiku 5.5 locally ๐
Hereโs what it had to say about the new model:
โTencent just put the model from Monday's deleted announcement on Hugging Face:
โข 220B total parameters, but only ~12B active per token, so it's cheap to run
โข Reads text and images, with a 1M-token context window
โข In Tencent's blind test with its own users, it scored 2.79 out of 4 vs 2.81 for Claude Haiku 5.5, at the lowest cost per task of any model tested
โข About 1/20 the cost per task of full Hy4
โข Also out: an FP8 version and the pre-trained base model
Definitely going to SAGE this one!! Stay tuned ๐
huggingface.co/tencent/Hy4-fโฆ
Tencent may be cooking with this one...
Hy4 Flash on a single DGX Spark: Ripple 65B quality at nearly double the speed ๐
I've spent the day putting Tencent's Hy4 Flash through its paces on one Spark, against Ripple 65B and the two Qwen3.8 open weight models.
Quality
Ripple 65B set the new standard on coding quality on a single Nvidia DGX Spark. Hy4 scored 76% on my 75-task reasoning eval, one point behind Ripple's 77% and well ahead of Qwen3.8 Flash-Next (63%) and 27B (56%).
Efficiency
This is the real killer feature - it gets there in about half the tokens, 11.7k per reply vs 22.6k for Ripple and roughly 25k for the Qwen models. Shorter replies plus faster decode means answers land much sooner.
Speed
up to 47.8 tok/s decode on code and 32 to 34 tok/s on prose, with prefill holding 846 tok/s at 2k and still 639 tok/s at 128k.
How it fits
Only 11.3B of its 155.5B parameters fire per token. The 63.5B n-gram memory is offloaded to SSD rather than sitting in VRAM, so the whole thing runs on one Spark with 295K context.
i'm 100% convinced...
that uncensored AI is the "LINUX" moment for AI.
it runs with or without a GPU, refuses nothing, and your data never leaves your machine.
here's how to build and run your OWN in 10 minutes โ
Ivan Rocha retweeted
Gemini 4 Neon: frontier-level intelligence with fast end-to-end task completion across complex software engineering, knowledge work, and science workflows.
Begins rolling out today in the Gemini API in @GoogleAIStudio, Gemini for business, for paid Google AI Pro and Ultra subscribers in Google @Antigravity, and for Google AI Pro and Ultra subscribers in the @GeminiApp.
A free online book: LLM Inference Handbook by Sherlock Xu (2026).
This book is a practical technical reference for engineers deploying, scaling, and operating language models in production.
It brings together the concepts and decisions involved in turning a trained model into a responsive, cost-aware service, from model behavior and hardware selection to application interfaces and ongoing operations.
Readers can use it as a guided progression or as a lookup resource for specific engineering questions.
Read the book online on @ChapterPal: chapterpal.com/book/3fef12a8โฆ


































